Dmitry Chalyy, Yaroslavl State University, Yaroslavl

1. Ìåòîäîëîãèÿ âîñïðîèçâîäèìûõ èññëåäîâàíèé
èìèòàöèîííûõ ìîäåëåé è ðåàëèçàöèé
êîììóíèêàöèîííûõ ïðîòîêëîâ
Äìèòðèé Þðüåâè÷ ×àëûé
ßðîñëàâñêèé ãîñóäàðñòâåííûé óíèâåðñèòåò, ßðîñëàâëü
13 íîÿáðÿ 2014 ã.
×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 1 / 8

2. Èñòî÷íèê: Google Ngram Viewer. Äàòà äîñòóïà: 12.11.2014.
url: https://books.google.com/ngrams
×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 2 / 8

3. Ïðèìèòèâíûé ïîäõîä ê âîñïðîèçâîäèìîñòè
Ìîæíî ëè îòêîìïèëèðîâàòü è çàïóñòèòü èñõîäíûé êîä
ýêñïåðèìåíòà çà 30 ìèíóò?
1 Äîñòóïåí ëè êîä è áèáëèîòåêè îò êîòîðûõ îí çàâèñèò?
2 Êîìïèëèðóåòñÿ? Çàïóñêàåòñÿ?
1Äîñòóïåí ïî àäðåñó: http://reproducibility.cs.arizona.edu/v1/tr.pdf. Äàòà
äîñòóïà: 12.11.2014. Âåðñèÿ äîêóìåíòà îò 21 ìàðòà 2014 ã.
×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 3 / 8

4. Ïðèìèòèâíûé ïîäõîä ê âîñïðîèçâîäèìîñòè
Ìîæíî ëè îòêîìïèëèðîâàòü è çàïóñòèòü èñõîäíûé êîä
ýêñïåðèìåíòà çà 30 ìèíóò?
1 Äîñòóïåí ëè êîä è áèáëèîòåêè îò êîòîðûõ îí çàâèñèò?
2 Êîìïèëèðóåòñÿ? Çàïóñêàåòñÿ?
Christian Collberg, Todd Proebsting, Gina Moraila, Akash
Shankaran, Zuoming Shi, Alex M. Warren Measuring
Reproducibility in Computer Systems Research1
1Äîñòóïåí ïî àäðåñó: http://reproducibility.cs.arizona.edu/v1/tr.pdf. Äàòà
äîñòóïà: 12.11.2014. Âåðñèÿ äîêóìåíòà îò 21 ìàðòà 2014 ã.
×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 3 / 8

5. Ïðèìèòèâíûé ïîäõîä ê âîñïðîèçâîäèìîñòè
Ìîæíî ëè îòêîìïèëèðîâàòü è çàïóñòèòü èñõîäíûé êîä
ýêñïåðèìåíòà çà 30 ìèíóò?
1 Äîñòóïåí ëè êîä è áèáëèîòåêè îò êîòîðûõ îí çàâèñèò?
2 Êîìïèëèðóåòñÿ? Çàïóñêàåòñÿ?
Christian Collberg, Todd Proebsting, Gina Moraila, Akash
Shankaran, Zuoming Shi, Alex M. Warren Measuring
Reproducibility in Computer Systems Research1
Ìåòîäîëîãèÿ èññëåäîâàíèÿ: àíàëèç 613 ñòàòåé (ASPLOS’12,
CCS’12, OOPSLA’12, OSDI’12, PLDI’12, SIGMOD’12, SOSP’11,
TACO’9, TISSEC’15, TOCS’30, TODS’37, TOPLAS’34,
VLDB’12).
1Äîñòóïåí ïî àäðåñó: http://reproducibility.cs.arizona.edu/v1/tr.pdf. Äàòà
äîñòóïà: 12.11.2014. Âåðñèÿ äîêóìåíòà îò 21 ìàðòà 2014 ã.
×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 3 / 8

6. Âîñïðîèçâîäèìîñòü â íàñòîÿùåå âðåìÿ
Èòîã: âñåãî 25% ðåçóëüòàòîâ êîìïèëèðóþòñÿ (èç êîòîðûõ
êàæäûé ÷åòâåðòûé ïðîåêò íå çàïóñêàåòñÿ)2
2Èñòî÷íèê: http://reproducibility.cs.arizona.edu/v1/index.html. Äàòà
äîñòóïà: 12.11.2014.
×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 4 / 8

7. Ïðîöåññ èññëåäîâàíèé

8. ×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 5 / 8

9. Ïðèìåð: òðàíñïîðòíûå ïðîòîêîëû. Èñõîäíûå äàííûå
1 Îðèãèíàëüíûå ìîäåëè (ARTCP).
2 Ïàêåòû èìèòàöèîííîãî ìîäåëèðîâàíèÿ: ns-3, OMNeT++,
Cisco Packet Tracer (TCP TIPS).
3 Çàõâàò ñåòåâîãî òðàôèêà íåïîñðåäñòâåííî ñ ñåòåâîãî
óñòðîéñòâà: tcpdump (îðèãèíàëüíûé Trickles).
×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 6 / 8

10. Ïðèìåð: òðàíñïîðòíûå ïðîòîêîëû. Àíàëèç è îáðàáîòêà
äàííûõ
1 Ñêðèïòîâûå ÿçûêè ïðîãðàììèðîâàíèÿ (shell, awk è ò.ä.).
2 R.
3 Âîçìîæíîñòè ñèñòåì ìîäåëèðîâàíèÿ (íàïð. ns-3 statistical
framework, wireshark).
×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 7 / 8

11. Ïðîåêòíûé ïîäõîä
1 Êàæäîå èññëåäîâàíèå - ýòî ïðîåêò.
2 Êàê ìîæíî ìåíüøå íåñòàíòàðäíûõ ñðåäñòâ ðàçðàáîòêè.
3 Îáùåäîñòóïíûé ðåïîçèòîðèé êîäà - bitbucket, github.
4 Äîêóìåíòàöèÿ - Doxygen.
5 Àíàëèç äàííûõ - R+{Sweave, knitr}+RPubs.
×àëûé Ä.Þ.: Âîñïðîèçâîäèìûå èññëåäîâàíèÿ êîììóíèêàöèîííûõ ïðîòîêîëîâ 8 / 8