Code review is the manual assessment of source code by humans, mainly intended to identify defects and quality problems. Modern Code Review (MCR), a lightweight variant of the code inspections investigated since the 1970s, prevails today both in industry and open-source software (OSS) systems. In this talk, I outlined the practical benefits that the MCR process produces on reviewed source code. To that end, I present two case studies that we carried out on two OSS systems each, classifying more than 1,400 review-induced source code changes manually. Surprisingly, results show that the types of changes due to the MCR process in OSS are strikingly similar to those in the industry and academic systems from literature, featuring the similar 75:25 ratio of maintainability-related to functional problems. I also reveal that 7–35% of review comments are discarded and that 10–22% of the changes are not triggered by an explicit review comment. Patterns emerged in the review data; we investigated them to get to know the technical factors that influence the number of changes due to the MCR process. We found that bug-fixing tasks lead to fewer changes and tasks with more altered files and a higher code churn have more changes. Contrary to intuition, the person of the reviewer had no impact on the number of changes.

1. Modern Code Reviews in Open-Source Projects:
Which Problems Do They Fix?
Moritz Beller
Delft University of Technology
@Inventitech
Alberto Bacchelli, Andy Zaidman
Elmar Juergens,
CQSE GmbH

2. Modern Code Review Workflow

3. Modern Code Review Workflow

4. Modern Code Review Workflow

5. Modern Code Review Workflow

6. Modern Code Review Workflow

7. Modern Code Review Workflow

8. Modern Code Review Workflow

9. Modern Code Review Workflow

10. Modern Code Review Workflow

11. Modern Code Review Workflow

12. Modern Code Review Workflow

13. Modern Code Review Workflow
+

15. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();

16. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();

17. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();

18. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
public Runnable watchDogUiThread = new
StartupUIThread();

19. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
public Runnable watchDogUiThread = new
StartupUIThread();

20. Review-triggered Change
/** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
public Runnable watchDogUiThread = new
StartupUIThread();

21. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();

22. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();

23. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
private Runnable watchDogUiThread = new
StartupUIThread();

24. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
private Runnable watchDogUiThread = new
StartupUIThread();

25. Self-motivated + Review-triggered Change
/** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
private Runnable watchDogUiThread = new
StartupUIThread();

26. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();

27. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();

28. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();
/**
* The ui thread for WatchDog registration. From
* this thread, the WatchDog program is started.
*/
public Runnable watchDog = new StartupUIThread();

29. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();
/**
* The ui thread for WatchDog registration. From
* this thread, the WatchDog program is started.
*/
public Runnable watchDog = new StartupUIThread();

30. Self-motivated Change
/** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();
/**
* The ui thread for WatchDog registration. From
* this thread, the WatchDog program is started.
*/
public Runnable watchDog = new StartupUIThread();

31. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();

32. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();

33. /** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();

34. No Change
/** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();

35. No Change
/** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();

36. No Change
/** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
// TODO (GG) Provide a more descriptive name
public Runnable watchDog = new StartupUIThread();
/** The ui thread for WatchDog registration. */
public Runnable watchDog = new StartupUIThread();

37. Does it matter?

38. >1,400 changes
>245 tasks

39. Motivation for Change

40. Motivation for Change

41. 78-90%
Motivation for Change

42. 78-90%
Motivation for Change
22-10%

43. 78-90%
Motivation for Change
22-10%

44. Motivation for No Change

45. Motivation for No Change

46. Motivation for No Change
7-35%

47. Motivation for No Change
7-35%

48. Type of Change

49. Doc
Visual
Structure
Resource
Check
Interface
Logic
Larger
Support
ConQAT
Gromacs
10% 20% 30% 40%

50. 75
:
25
Doc
Visual
Structure
Resource
Check
Interface
Logic
Larger
Support
ConQAT
Gromacs
10% 20% 30% 40%

51. 75
:
25
Doc
Visual
Structure
Resource
Check
Interface
Logic
Larger
Support
ConQAT
Gromacs
10% 20% 30% 40%

52. 75
:
25
Doc
Visual
Structure
Resource
Check
Interface
Logic
Larger
Support
ConQAT
Gromacs
10% 20% 30% 40%

54. 75 : 25

55. What Influences Code Review?

56. What Influences Code Review?

57. What Influences Code Review?

58. What Influences Code Review?
> 2,800 changes
> 970 tasks

59. What Influences Code Review?
> 2,800 changes
> 970 tasks

60. What Influences Code Review?
> 2,800 changes
> 970 tasks

61. What Influences Code Review?
> 2,800 changes
> 970 tasks

62. What Influences Code Review?
> 2,800 changes
> 970 tasks

63. What Influences Code Review?
> 2,800 changes
> 970 tasks

64. What Influences Code Review?
> 2,800 changes
> 970 tasks

65. What Influences Code Review?
> 2,800 changes
> 970 tasks

66. @Inventitech
Moritz Beller, TU Delft
Modern Code Reviews in Open-Source Projects:
Which Problems Do They Fix?

67. @Inventitech
Moritz Beller, TU Delft
Modern Code Reviews in Open-Source Projects:
Which Problems Do They Fix?

68. @Inventitech
Moritz Beller, TU Delft
Modern Code Reviews in Open-Source Projects:
Which Problems Do They Fix?

69. @Inventitech
Moritz Beller, TU Delft
Modern Code Reviews in Open-Source Projects:
Which Problems Do They Fix?