The document discusses ways to objectively measure how good of a software developer someone is. It reviews literature on using aptitude tests, education levels, experience, programming knowledge, and skill to predict job performance. Skill is identified as a key predictor that improves with practice. The document then discusses models of skill acquisition and outlines research measuring programming skill through tasks that combine time and quality metrics. It proposes using a Rasch model to create a measurement system for skills by having developers complete standardized tasks that are scored against a large control group.
How good of developer are you v1.1 DevWorld.pdfRoy Wasse
This document discusses different approaches for measuring how good of a software developer someone is. It reviews literature showing that technical skill is the most important factor, but other dimensions like knowledge, experience, and motivation also contribute to job performance. Task performance on coding assignments is identified as the key way to objectively measure programming skill. The document proposes developing a standardized measurement model based on analyzing time and quality of performance on coding tasks, validated against a large sample of developers, to reliably predict skills.
Applying AI to software engineering problems: Do not forget the human!University of Córdoba
The application of artificial intelligence (AI) to software engineering (SE)-problem-solving has been around since the 80s when expert systems were first used. However, it is during the last 10 years that there has been a peak in the use of these techniques, first based on search and optimisation algorithms such as metaheuristics, and later based on machine learning algorithms. The aim is to help the software engineer to automate and optimise tasks of the software development process, and to use valuable information hidden in multiple data sources such as software repositories to execute insightful actions that generate improvements in the performance of the overall process. Today, the use of AI is trendy, and often overused as it could generate artificial results since it does not consider the subjective nature of the software development process requiring the experience and know-how of the engineer. With this Invited Talk, we will discuss different proposals to incorporate the human into the decision-making process in the application of AI for SE (AI4SE), from interactive algorithms to the generation of interpretable models or explanations.
This document discusses different perspectives on software development:
- Software development is best viewed as a cooperative game of invention and communication, where the goal is to deliver working software while preparing for future projects.
- Considering it as engineering or model building does not capture the creative, communicative nature of the work.
- Programming involves building a theory of the problem and solution domain through discussion and design, not just producing documentation and code.
Software development is best understood as a cooperative game of invention and communication, rather than an engineering process or model building exercise. The primary goal of the game is to deliver useful working software, while the secondary goal is to set up for future projects. Successful software development relies more on the ideas and discussions between team members than on formal work products and models. Programmers function more as communication specialists than non-communicative coders. Building a theory of the problem and solution through communication is more important than documentation.
Software Engineering: why it is more than coding, and why it is necessaryYazid Hamdi
Software engineering involves more than just coding. It is a set of practices that help ensure software projects are completed on time, on budget, and meet requirements. These practices include requirements engineering, planning and tracking, architecture design, configuration management, quality management, and risk management. Following a disciplined software engineering process helps reduce risks and issues that can arise from an ad-hoc development approach. The key is to have documented processes, collect feedback to improve over time, and automate processes when possible.
Web-Based Self- and Peer-Assessment of Teachers’ Educational Technology Compe...Hans Põldoja
This document summarizes a research project that aimed to develop a web-based tool for assessing teachers' educational technology competencies through self-assessment and peer assessment. It outlines existing competency frameworks, the design challenges, methodology used which included personas, scenarios, and participatory design sessions. Prototypes were created including a competency test, profile, grouping and requirements features. Future work includes expanding assessment tasks and integrating the tool into other digital platforms.
The complimentary deck to my public speech on "People vs Process" in Minsk at Jan-2020. It's contains the intermediate results of research I'm working on.
Inside is description of the approach to non-invasive measurements of software developers productivity.
For more detailed info please contact alexander_nemtsov at gmail.com
The document discusses ways to objectively measure how good of a software developer someone is. It reviews literature on using aptitude tests, education levels, experience, programming knowledge, and skill to predict job performance. Skill is identified as a key predictor that improves with practice. The document then discusses models of skill acquisition and outlines research measuring programming skill through tasks that combine time and quality metrics. It proposes using a Rasch model to create a measurement system for skills by having developers complete standardized tasks that are scored against a large control group.
How good of developer are you v1.1 DevWorld.pdfRoy Wasse
This document discusses different approaches for measuring how good of a software developer someone is. It reviews literature showing that technical skill is the most important factor, but other dimensions like knowledge, experience, and motivation also contribute to job performance. Task performance on coding assignments is identified as the key way to objectively measure programming skill. The document proposes developing a standardized measurement model based on analyzing time and quality of performance on coding tasks, validated against a large sample of developers, to reliably predict skills.
Applying AI to software engineering problems: Do not forget the human!University of Córdoba
The application of artificial intelligence (AI) to software engineering (SE)-problem-solving has been around since the 80s when expert systems were first used. However, it is during the last 10 years that there has been a peak in the use of these techniques, first based on search and optimisation algorithms such as metaheuristics, and later based on machine learning algorithms. The aim is to help the software engineer to automate and optimise tasks of the software development process, and to use valuable information hidden in multiple data sources such as software repositories to execute insightful actions that generate improvements in the performance of the overall process. Today, the use of AI is trendy, and often overused as it could generate artificial results since it does not consider the subjective nature of the software development process requiring the experience and know-how of the engineer. With this Invited Talk, we will discuss different proposals to incorporate the human into the decision-making process in the application of AI for SE (AI4SE), from interactive algorithms to the generation of interpretable models or explanations.
This document discusses different perspectives on software development:
- Software development is best viewed as a cooperative game of invention and communication, where the goal is to deliver working software while preparing for future projects.
- Considering it as engineering or model building does not capture the creative, communicative nature of the work.
- Programming involves building a theory of the problem and solution domain through discussion and design, not just producing documentation and code.
Software development is best understood as a cooperative game of invention and communication, rather than an engineering process or model building exercise. The primary goal of the game is to deliver useful working software, while the secondary goal is to set up for future projects. Successful software development relies more on the ideas and discussions between team members than on formal work products and models. Programmers function more as communication specialists than non-communicative coders. Building a theory of the problem and solution through communication is more important than documentation.
Software Engineering: why it is more than coding, and why it is necessaryYazid Hamdi
Software engineering involves more than just coding. It is a set of practices that help ensure software projects are completed on time, on budget, and meet requirements. These practices include requirements engineering, planning and tracking, architecture design, configuration management, quality management, and risk management. Following a disciplined software engineering process helps reduce risks and issues that can arise from an ad-hoc development approach. The key is to have documented processes, collect feedback to improve over time, and automate processes when possible.
Web-Based Self- and Peer-Assessment of Teachers’ Educational Technology Compe...Hans Põldoja
This document summarizes a research project that aimed to develop a web-based tool for assessing teachers' educational technology competencies through self-assessment and peer assessment. It outlines existing competency frameworks, the design challenges, methodology used which included personas, scenarios, and participatory design sessions. Prototypes were created including a competency test, profile, grouping and requirements features. Future work includes expanding assessment tasks and integrating the tool into other digital platforms.
The complimentary deck to my public speech on "People vs Process" in Minsk at Jan-2020. It's contains the intermediate results of research I'm working on.
Inside is description of the approach to non-invasive measurements of software developers productivity.
For more detailed info please contact alexander_nemtsov at gmail.com
This document summarizes the findings of a study that interviewed 59 experienced software engineers at Microsoft to understand the attributes of great software engineers. The study identified 53 attributes that were organized into internal personal characteristics and abilities of engineers, as well as external impacts engineers have on people, products, and users. Some of the most important internal attributes included being passionate about their work, continuously improving skills, and having strong practical decision-making abilities. Externally, great engineers applied their skills to create high-quality, elegant software and formed trusting relationships with teammates. The study provided insights and definitions for each attribute based on the interviews.
This document summarizes Zohreh Sharafi's research on the influence of representation type and gender on program comprehension. The research includes several experiments that investigate how representation type (graphical vs. textual) and gender influence developers' efficiency, effectiveness, and viewing strategies during program comprehension tasks. The results show that developers prefer graphical representations and find relevant information faster with graphical representations compared to textual. The experiments also found that men and women use different strategies to select correct answers, with women taking longer but achieving higher accuracy. Further analysis revealed differences in how men and women distribute their visual attention across source code entities during comprehension tasks.
This document provides an introduction to software engineering. It discusses how software serves both as a product that delivers computing potential and as a vehicle for delivering other products. The document defines what constitutes software and discusses different types of software applications. It also covers software engineering practices, including communication, planning, analysis and design modeling, construction, and principles related to each practice. Overall, the document gives a high-level overview of key concepts in software engineering.
The document provides an introduction to software engineering. It discusses that software has a dual role as both a product and vehicle to deliver functionality. It defines software as a set of programs, documents, and data that form a configuration. The document outlines different types of software applications and categories. It also discusses software engineering practices such as communication, planning, modeling, construction, and coding principles.
The document discusses data-driven approaches to optimizing software testing processes at Microsoft. It describes how historical test and code data can be analyzed to determine which tests are most valuable and cost-effective to run, in order to reduce total test execution time without negatively impacting code quality. Simulation results on Windows 8.1 data show the potential for significant test reduction (up to 60%) while maintaining bug finding ability. This could improve development processes by lowering machine costs and increasing developer satisfaction.
This document provides advice on how non-profits can build strong software engineering teams through an effective hiring process. It discusses challenges non-profits face in hiring, such as lower salaries compared to industry. It recommends casting a wide net to find candidates, running a rigorous interview process to filter candidates, hiring for potential as well as experience, assessing organizational fit, and making strong offers that emphasize mission over salary alone. The document provides examples from the author's experience hiring a new tech team at MoveOn.org, highlighting the importance of an inclusive job description and interview process for finding diverse talent.
This document discusses software engineering and provides definitions and explanations of key concepts:
- Software engineering is defined as an engineering discipline concerned with all aspects of software production. It focuses on practical software development and delivery, whereas computer science focuses more on theory.
- Good software should deliver required functionality, performance, and be maintainable, dependable, usable and acceptable to users.
- A software engineering approach is layered, with quality, process models, methods and tools. Process models define activities for effective delivery. Methods provide tasks for requirements, design, coding and testing. Tools support the process and methods.
- Generic software processes involve communication, planning, modeling, construction and deployment activities in an iterative fashion to develop
This paper systematically reviewed 46 publications on the impact of global dispersion on software development coordination, team performance, and quality. It found that geographical dispersion can decrease communication and increase coordination challenges. Cultural dispersion often causes misunderstandings while temporal dispersion introduces time lags. Uneven work distributions across locations also negatively impacted quality. However, temporal dispersion was also found to increase development speed in some cases. The paper discusses implications for managers in balancing costs and coordination challenges of dispersed teams.
Critiquing CS Assessment from a CS for All lens: Dagstuhl Seminar PosterMark Guzdial
Poster presented at the Dagstuhl Seminar "Assessing Learning in Introductory Computer Science" (http://www.dagstuhl.de/en/program/calendar/semhp/?semnr=16072). I argue that we have to consider what the learner wants to do and wants to be (i.e., their desired Community of Practice) when assessing learning. Different CoP, different outcomes, different assessments.
The goal of this presentation is to give attendees a deeper understanding of usability testing so they can leverage it in their own work. The material will shed light on what is important to the research buyer and will help the research provider to better understand how to plan, moderate, and report on a usability study. It will also provide information on where they can go to learn more about this very practical qualitative method.
Kay will cover what a usability test is and when to use it, the key planning steps, the language around it, and the unique insights this method produces. She will also discuss the various approaches a market researcher can take when running a usability study at different points in a product’s development (e.g., concept, early prototype, released product).
This document discusses prescriptive and agile software process models. Prescriptive models emphasize detailed planning to improve quality, manageability, and predictability. However, they can increase bureaucracy if applied rigidly. Agile models prioritize adaptability and are useful for web applications. The document also discusses Polya's problem-solving steps of understanding the problem, planning a solution, carrying out the plan, and examining the results, as well as general principles and myths about software engineering.
Test-Driven Development in the Corporate WorkplaceAhmed Owian
What is TDD, and why is it giving traditional software development practices a run for their money? This presentation answers these questions, while focusing on a popular agile methodology, Extreme Programming (XP). It places a particular emphasis on the exploratory programming nature of XP and its testing practice, TDD. The paper also summarizes prior research on TDD and includes the results from a research survey conducted to compare TDD with traditional testing practices.
AI improves software testing through test automation, test creation and test ...Kari Kakkonen
The early 2024 version of my How AI improve testing presentation that I delivery since 5 years ago in different conference, webinars and customer events abroad and in Finland.
It gives some background on AI and talks about AI opportunities: in short test automation getting more fault tolerant, generative AI bringing efficiency to test creation, and advanced test selection with AI.
AI improves software testing to be more fault tolerant, focused and efficientKari Kakkonen
My slides as Finnish Testing Meetup 28.2.2024
AI supports testing in multiple ways: test automation becomes more autonomous and fault-tolerant, test selection is easier after AI analysis and recommendation, and manual and test automation productivity goes up with generative AI. Some background too on AI in general.
The document discusses key practices in software engineering including communication practices, planning practices, analysis and design modeling practices, construction practices, testing practices, and deployment practices. It provides principles for each practice such as keeping communication focused and documented, estimating realistically, designing for simplicity and traceability, testing exhaustively, and managing customer expectations during deployment. The overall aim is to transform an unfocused approach into one that is organized, effective, and achieves success.
Addressing learning gaps and career oppurtunities after B.Sc computer sciencesandhya12bansal
Addressees the learning gaps i.e why after graduation in B.Sc computer science student is still unemployed. PPts discusses various examples for learning methods. The second part of the presentation discusses the various opportunities after B.Sc in Computer science
This document discusses whether big data analysis is more of a "systems" task or "human" task. It presents research showing that software defect prediction, even when conducted by top experts using the same datasets and algorithms over many years, shows little improvement and high variability. This suggests that human factors like biases are important. The document proposes using data mining on source code and social media to classify developers by expertise and identify groups who could share knowledge to reduce defects. It outlines an initial approach using parsers, classifiers like Naive Bayes to distinguish novices from experts, and seeking larger datasets from partners. The goal is to strengthen the "human" aspects of big data analysis.
Bridging the Gap Between Data Science & Engineer: Building High-Performance T...ryanorban
Data scientists, data engineers, and data businesspeople are critical to leveraging data in any organization. A common complaint from data science managers is that data scientists invest time prototyping algorithms, and throw them over a proverbial fence to engineers to implement, only to find the algorithms must be rebuilt from scratch to scale. This is a symptom of a broader ailment -- that data teams are often designed as functional silos without proper communication and planning.
This talk outlines a framework to build and organize a data team that produces better results, minimizes wasted effort among team members, and ships great data products.
The document discusses modern software development approaches and best practices. It covers topics like popular programming languages, architectural patterns, team structures, and practices. Key points include: JavaScript, Python, and Java are the most popular languages; architectural approaches like monoliths, microservices, and serverless all have tradeoffs; modern teams include roles like SREs and benefit from practices like living documentation and mob programming; and factors like team maturity impact project success rates more than project size alone.
Weitere ähnliche Inhalte
Ähnlich wie How good of developer are you v1 .1 ASML.pptx
This document summarizes the findings of a study that interviewed 59 experienced software engineers at Microsoft to understand the attributes of great software engineers. The study identified 53 attributes that were organized into internal personal characteristics and abilities of engineers, as well as external impacts engineers have on people, products, and users. Some of the most important internal attributes included being passionate about their work, continuously improving skills, and having strong practical decision-making abilities. Externally, great engineers applied their skills to create high-quality, elegant software and formed trusting relationships with teammates. The study provided insights and definitions for each attribute based on the interviews.
This document summarizes Zohreh Sharafi's research on the influence of representation type and gender on program comprehension. The research includes several experiments that investigate how representation type (graphical vs. textual) and gender influence developers' efficiency, effectiveness, and viewing strategies during program comprehension tasks. The results show that developers prefer graphical representations and find relevant information faster with graphical representations compared to textual. The experiments also found that men and women use different strategies to select correct answers, with women taking longer but achieving higher accuracy. Further analysis revealed differences in how men and women distribute their visual attention across source code entities during comprehension tasks.
This document provides an introduction to software engineering. It discusses how software serves both as a product that delivers computing potential and as a vehicle for delivering other products. The document defines what constitutes software and discusses different types of software applications. It also covers software engineering practices, including communication, planning, analysis and design modeling, construction, and principles related to each practice. Overall, the document gives a high-level overview of key concepts in software engineering.
The document provides an introduction to software engineering. It discusses that software has a dual role as both a product and vehicle to deliver functionality. It defines software as a set of programs, documents, and data that form a configuration. The document outlines different types of software applications and categories. It also discusses software engineering practices such as communication, planning, modeling, construction, and coding principles.
The document discusses data-driven approaches to optimizing software testing processes at Microsoft. It describes how historical test and code data can be analyzed to determine which tests are most valuable and cost-effective to run, in order to reduce total test execution time without negatively impacting code quality. Simulation results on Windows 8.1 data show the potential for significant test reduction (up to 60%) while maintaining bug finding ability. This could improve development processes by lowering machine costs and increasing developer satisfaction.
This document provides advice on how non-profits can build strong software engineering teams through an effective hiring process. It discusses challenges non-profits face in hiring, such as lower salaries compared to industry. It recommends casting a wide net to find candidates, running a rigorous interview process to filter candidates, hiring for potential as well as experience, assessing organizational fit, and making strong offers that emphasize mission over salary alone. The document provides examples from the author's experience hiring a new tech team at MoveOn.org, highlighting the importance of an inclusive job description and interview process for finding diverse talent.
This document discusses software engineering and provides definitions and explanations of key concepts:
- Software engineering is defined as an engineering discipline concerned with all aspects of software production. It focuses on practical software development and delivery, whereas computer science focuses more on theory.
- Good software should deliver required functionality, performance, and be maintainable, dependable, usable and acceptable to users.
- A software engineering approach is layered, with quality, process models, methods and tools. Process models define activities for effective delivery. Methods provide tasks for requirements, design, coding and testing. Tools support the process and methods.
- Generic software processes involve communication, planning, modeling, construction and deployment activities in an iterative fashion to develop
This paper systematically reviewed 46 publications on the impact of global dispersion on software development coordination, team performance, and quality. It found that geographical dispersion can decrease communication and increase coordination challenges. Cultural dispersion often causes misunderstandings while temporal dispersion introduces time lags. Uneven work distributions across locations also negatively impacted quality. However, temporal dispersion was also found to increase development speed in some cases. The paper discusses implications for managers in balancing costs and coordination challenges of dispersed teams.
Critiquing CS Assessment from a CS for All lens: Dagstuhl Seminar PosterMark Guzdial
Poster presented at the Dagstuhl Seminar "Assessing Learning in Introductory Computer Science" (http://www.dagstuhl.de/en/program/calendar/semhp/?semnr=16072). I argue that we have to consider what the learner wants to do and wants to be (i.e., their desired Community of Practice) when assessing learning. Different CoP, different outcomes, different assessments.
The goal of this presentation is to give attendees a deeper understanding of usability testing so they can leverage it in their own work. The material will shed light on what is important to the research buyer and will help the research provider to better understand how to plan, moderate, and report on a usability study. It will also provide information on where they can go to learn more about this very practical qualitative method.
Kay will cover what a usability test is and when to use it, the key planning steps, the language around it, and the unique insights this method produces. She will also discuss the various approaches a market researcher can take when running a usability study at different points in a product’s development (e.g., concept, early prototype, released product).
This document discusses prescriptive and agile software process models. Prescriptive models emphasize detailed planning to improve quality, manageability, and predictability. However, they can increase bureaucracy if applied rigidly. Agile models prioritize adaptability and are useful for web applications. The document also discusses Polya's problem-solving steps of understanding the problem, planning a solution, carrying out the plan, and examining the results, as well as general principles and myths about software engineering.
Test-Driven Development in the Corporate WorkplaceAhmed Owian
What is TDD, and why is it giving traditional software development practices a run for their money? This presentation answers these questions, while focusing on a popular agile methodology, Extreme Programming (XP). It places a particular emphasis on the exploratory programming nature of XP and its testing practice, TDD. The paper also summarizes prior research on TDD and includes the results from a research survey conducted to compare TDD with traditional testing practices.
AI improves software testing through test automation, test creation and test ...Kari Kakkonen
The early 2024 version of my How AI improve testing presentation that I delivery since 5 years ago in different conference, webinars and customer events abroad and in Finland.
It gives some background on AI and talks about AI opportunities: in short test automation getting more fault tolerant, generative AI bringing efficiency to test creation, and advanced test selection with AI.
AI improves software testing to be more fault tolerant, focused and efficientKari Kakkonen
My slides as Finnish Testing Meetup 28.2.2024
AI supports testing in multiple ways: test automation becomes more autonomous and fault-tolerant, test selection is easier after AI analysis and recommendation, and manual and test automation productivity goes up with generative AI. Some background too on AI in general.
The document discusses key practices in software engineering including communication practices, planning practices, analysis and design modeling practices, construction practices, testing practices, and deployment practices. It provides principles for each practice such as keeping communication focused and documented, estimating realistically, designing for simplicity and traceability, testing exhaustively, and managing customer expectations during deployment. The overall aim is to transform an unfocused approach into one that is organized, effective, and achieves success.
Addressing learning gaps and career oppurtunities after B.Sc computer sciencesandhya12bansal
Addressees the learning gaps i.e why after graduation in B.Sc computer science student is still unemployed. PPts discusses various examples for learning methods. The second part of the presentation discusses the various opportunities after B.Sc in Computer science
This document discusses whether big data analysis is more of a "systems" task or "human" task. It presents research showing that software defect prediction, even when conducted by top experts using the same datasets and algorithms over many years, shows little improvement and high variability. This suggests that human factors like biases are important. The document proposes using data mining on source code and social media to classify developers by expertise and identify groups who could share knowledge to reduce defects. It outlines an initial approach using parsers, classifiers like Naive Bayes to distinguish novices from experts, and seeking larger datasets from partners. The goal is to strengthen the "human" aspects of big data analysis.
Bridging the Gap Between Data Science & Engineer: Building High-Performance T...ryanorban
Data scientists, data engineers, and data businesspeople are critical to leveraging data in any organization. A common complaint from data science managers is that data scientists invest time prototyping algorithms, and throw them over a proverbial fence to engineers to implement, only to find the algorithms must be rebuilt from scratch to scale. This is a symptom of a broader ailment -- that data teams are often designed as functional silos without proper communication and planning.
This talk outlines a framework to build and organize a data team that produces better results, minimizes wasted effort among team members, and ships great data products.
Ähnlich wie How good of developer are you v1 .1 ASML.pptx (20)
The document discusses modern software development approaches and best practices. It covers topics like popular programming languages, architectural patterns, team structures, and practices. Key points include: JavaScript, Python, and Java are the most popular languages; architectural approaches like monoliths, microservices, and serverless all have tradeoffs; modern teams include roles like SREs and benefit from practices like living documentation and mob programming; and factors like team maturity impact project success rates more than project size alone.
Roy Wasse presented an overview of cryptography concepts including classical ciphers like the Caesar cipher and Vigenère cipher. Modern symmetric ciphers use pseudorandom number generators with nonces to prevent key reuse. Asymmetric encryption allows public/private key pairs. Quantum computing may break current algorithms like RSA and ECC. Anonymity can be achieved using mixing services and onion routing. Zero-knowledge proofs allow verification without revealing information. New developments like Schnorr signatures and Mimblewimble aim to improve scalability and privacy in cryptocurrencies.
This document provides an overview of cryptography concepts including various ciphers like the Caesar cipher, Vigenère cipher, Playfair cipher, and the one-time pad. It discusses stream ciphers using techniques like the middle squares method. Asymmetric encryption techniques like elliptic curve cryptography are covered as well as the impact of quantum computing. Additional topics include hashing, block ciphers, mixing services/onion routing, zero-knowledge proofs, and scriptless scripts. Recommended further reading on applied cryptography is also provided.
The document discusses the evolution of digital cash and blockchain technology. It covers key aspects of blockchain including peer-to-peer networks, distributed ledgers, consensus mechanisms, incentives, hashing, and smart contracts. Examples are provided of how blockchain can be used for payments without banks, guaranteeing message delivery, and preventing transaction manipulation. Ethereum is discussed as a platform for decentralized applications and smart contracts. Considerations for when blockchain is and is not suitable are also outlined.
Roy Wasse is a Dutch JUG leader and co-founder of OpenValue who is interested in technological change. The document discusses various cryptographic techniques including the one-time pad encryption method, stream ciphers, hashing versus encryption, block ciphers, asymmetric encryption using elliptic curves, quantum key distribution, and various applications like mixing services and onion routing for anonymity. It also touches on concepts like commitment schemes, zero-knowledge proofs, scriptless transactions in Mimblewimble, and using digital signatures to unlock content.
Ready to Unlock the Power of Blockchain!Toptal Tech
Imagine a world where data flows freely, yet remains secure. A world where trust is built into the fabric of every transaction. This is the promise of blockchain, a revolutionary technology poised to reshape our digital landscape.
Toptal Tech is at the forefront of this innovation, connecting you with the brightest minds in blockchain development. Together, we can unlock the potential of this transformative technology, building a future of transparency, security, and endless possibilities.
Gen Z and the marketplaces - let's translate their needsLaura Szabó
The product workshop focused on exploring the requirements of Generation Z in relation to marketplace dynamics. We delved into their specific needs, examined the specifics in their shopping preferences, and analyzed their preferred methods for accessing information and making purchases within a marketplace. Through the study of real-life cases , we tried to gain valuable insights into enhancing the marketplace experience for Generation Z.
The workshop was held on the DMA Conference in Vienna June 2024.
HijackLoader Evolution: Interactive Process HollowingDonato Onofri
CrowdStrike researchers have identified a HijackLoader (aka IDAT Loader) sample that employs sophisticated evasion techniques to enhance the complexity of the threat. HijackLoader, an increasingly popular tool among adversaries for deploying additional payloads and tooling, continues to evolve as its developers experiment and enhance its capabilities.
In their analysis of a recent HijackLoader sample, CrowdStrike researchers discovered new techniques designed to increase the defense evasion capabilities of the loader. The malware developer used a standard process hollowing technique coupled with an additional trigger that was activated by the parent process writing to a pipe. This new approach, called "Interactive Process Hollowing", has the potential to make defense evasion stealthier.
Discover the benefits of outsourcing SEO to Indiadavidjhones387
"Discover the benefits of outsourcing SEO to India! From cost-effective services and expert professionals to round-the-clock work advantages, learn how your business can achieve digital success with Indian SEO solutions.
1. How good of a
developer are you?
And can we objectively measure this?
2. Bio
• Roy Wasse
• Bsc. Technical
Computer Science
• Msc. Philosophy of
Technology
• Co Founder OpenValue
• Dev & Management roles
• 15+ years of hiring
experience
3. Is someone is a
‘good’ dev!
• Do you know YAGNI or HATEOS, SOLID?
• Explain balanced binary tree search algo?
• Use 20 best interview questions from
internet
• Apply Open Hiring?
• Proxy variables e.g., CV, experience,
education, references, questionnaire
• Whiteboard coding
• Make (lengthy) assignment
4. Google Hiring process
• Assessments: You may be asked to do a brief online assessment, like a coding quiz, after
you’ve submitted your resume.
• Short virtual chats: Before diving into more in-depth interviews, you’ll typically have
one or two shorter conversations over phone or video. These will usually be with a
recruiter and then with either the hiring manager or peer on the team, and are designed
to assess key skills you’ll need for the role.
• Project work: We sometimes ask candidates to complete a small project prior to their in-
depth interviews. This could range from prepping a case study to providing writing or
code samples (don’t stress, they’re not that scary and we won’t spring this on you
without warning), and helps us understand how you think and approach problems. We’ll let
you know about any additional materials we’ll need early on.
• Interviews: We get excited about interviewing and take it seriously because, at the risk
of sounding cliché, Google is what Googlers make it. Our process can be rigorous
(typically 3-4 interviews in one day, either over video or in person).
5. • Can science help us and
how?
• Let’s dive into
literature
• And see how we can
predict of someone will
be a good dev!
Who knows what
works?!!
6. Use of Aptitude tests
• Measure if someone is dispositioned well to excel
• Popular in the 1970s (e.g., Alspaugh, 1972; Mazlack,
1980)
• due to lack of good results from these tests not used
much anymore (Bornat, Dehnadi, & Simon, 2008; Curtis,
1991; Mayer & Stalnaker, 1968)
• Also: in a sample of over 3,500 students, no
incremental validity was found for specialized
aptitude tests for programmers over the more general
aptitude tests used by the United States Air Force
(Besetsny, Ree, & Earles, 1993)
7. Question
• Who thinks a computer science study is an
important predictor for the expert level of a
developer?
8. Role of
education
• Large meta study (Roth,
BeVier, Switzer, &
Schippmann, 1996)
reported a correlation of
r = 0.16
• Shortly after graduation,
correlation between
grades and job
performance much higher
• 1 year, r = 0.23,
n = 1,288
• 6 years, r = 0.05,
n = 866.
10. Role of
experience
• Studies show positive effect
of having programming
experience (Agarwal, Sinha &
Tanniru, 1996; Arisholm &
Sjøberg, 2004)
• Other studies have reported
no effect of experience
beyond the first few years
(Jeffery & Lawrence, 1979) or
no effect of experience
(Jørgensen, 1995; Wohlin,
2002, 2004)
11. Question
• Who thinks LLMs will replace us within the next
few years for the better part of our job?
12. Role of LLMs
• Refactoring vs. Refuctering (Tornhill, Borg, Mones, 2024)
• Ideal for who?
• Generation works pretty well, but what about code deletion?
13. Question
• Who thinks Java/.Net/... certification programs
are important for a developer?
14. Role of
Programming
knowledge
• Central predictor of skill is
knowledge (Chi et al., 1988)
• Used in Java certification
• But…software development is a
skill
Acquired by practice
15. What is skill?
• Ability (Ferguson, 1956):
“An ability is a trait
defined by what an
individual can do”
• “Skill is specialized type
of ability, one that
improves with practice and
is well adjusted, well
organized and goal
oriented” (Fitts & Posner,
1967)
16. Model of skill acquisition
• “A Five-Stage Model of the Mental Activities
Involved in Directed Skill Acquisition”,
(Dreyfus & Dreyfus, 1980)
• Flow is at level 5
• Also think of driving / playing chess
17. It’s all about skill?
• Skill cannot by equated with performance
• “the individual performed well because of his
high level of skill” is a generalization
(Messick, 1994)
• 8 major components of job performance research
show 3 direct determinants of performance on
the job: knowledge, skill and motivation
(Campbell, McCloy, Oppler & Sager, 1993)
18. Skill is just one..?
Campbell’s work performance framework proposed eight work performance
dimensions:
1. job-specific task proficiency <= skill
2. non–job-specific task proficiency
3. written and oral communications
4. demonstrating effort
5. maintaining personal discipline
6. facilitating peer and team performance
7. supervision
8. management and administration.
According to Campbell, these eight dimensions are sufficient to describe
the latent structure of performance at a general level. However, he also
noted that the eight factors can have different patterns of subdimensions,
and their content and salience can vary across jobs.
20. Skill & project success
emphasizing low price when selecting software providers in bidding
rounds increases the likelihood project will fail.” (Jørgensen 2011)
Tested by analyzing a dataset of 4,791,067 bids for 785,326 small-
scale projects
Clients emphasizing a low price selected providers with lower skill,
resulted in 9% increase in failure rate
21. What about
teamwork
• Team performance is more
complex to understand
than individual
performance (see, e.g.,
Baker & Salas, 1992;
Volmer, 2006),
individual skill or
expertise is
nevertheless a central
component in team
performance (Land, Wong,
& Jeffery, 2003).
22. Importance of developers
• The claim of substantial programming performance
differences on success is confirmed by additional
data, reviews & experiments (Curtis, 1981;
DeMarco & Lister, 1985, 1999; Trendowicz & Munch,
2009; Prechelt, 1999a)
• Variability in individual performance more
important than technology or methods used to
increase productivity
• In the book “Software Economics”, Boehm (1981)
reported that the “people factor” was the largest
of all investigated factors in the success of
software projects and therefore concluded that
developer attributes are by far the best
opportunity for improving software productivity.
23. Pair experts for extra speed
• The benefits of pair programming has been studied
from the perspective of forming the pair based on
different levels of expertise (Lui & Chan, 2006)
• …or seniority (Arisholm, Gallis, Dybå, & Sjøberg,
2007).
• A systematic literature review on pair programming
found that, among students, this practice was most
beneficial when the pair was comprised of
individuals with a similar level of programming
skill (Salleh, Mendes, & Grundy, 2011).
24. So common sense wins?
• A software developer is someone who usually
holds “a bachelor’s degree in computer science
and strong computer programming skills” (US
Bureau of Labor Statistics)
• So let’s hire based on skill level!!
27. Research overview
• Relations between effort estimates, skill indicators, and measured
programming skill. Magne Jørgensen, Gunnar R. Bergersen and Knut
Liestøl (2021)
• Measuring Programming Skill: Construction and Validation of an
Instrument for Evaluating Java Developers. Gunnar R. Bergersen,
PhD thesis (2015)
• Construction and validation of an instrument for measuring
programming skill. Gunnar R. Bergersen, Dag I. K. Sjøberg, and
Tore Dybå (2014)
• Evaluating methods and technologies in software engineering with
respect to developers’ skill level. Gunnar R. Bergersen and Dag I.
K. Sjøberg (2012)
• Inferring skill from tests of programming performance: combining
time and quality. Gunnar R. Bergersen, Jo E. Hannay, Dag I. K.
Sjøberg, Tore Dybå, and Amela Karahasanovic (2011)
• Programming skill, knowledge, and working memory among
professional developers from an investment theory perspective.
Gunnar R. Bergersen and Jan-Eric Gustafsson (2011)
28. How to measure skill?
• Home work assignment (better rotate those)
• Whiteboard coding (under pressure)
• Online questionaires (assuming correlation
between real world coding)
• All are subjective
• And probably you want to get it right!
29. Measure via self
assessment
• Ratings of self may work
better when people compare
themselves relative to
each other (people they
know) rather than when
providing absolute ratings
(Kruger and Dunning (1999)
Source:
wikipedia
30. Measuring programming skill
• Research started in 50’s with punch card
operators traits (McNamara & Hughes, 1955)
• 60’s use of test in selection researched (Perry
& Cantley, 1965)
• Behavorial study of devs (Shneiderman, 1976)
• Theories concerning the comprehension of
computer programs (Brooks, 1983)
31. Task performance is key
• >20 studies about job performance assessed
• Almost all frameworks mentioned task
performance as an important dimension of
individual work performance.
32. Predict skill
• Study of novice LISP
programmers (Anderson &
Jeffries, 1985) found that
“the best predictor of
individual subject
differences in errors on
problems that involved one
LISP concept was the number
of errors on other problems
that involved different
concepts” (Anderson, 1987,
p. 203).
33. How to put this into a system
• Time (Bergersen, 2011)
• Quality (Ebert et. al. 2005)
• Process improvement (Dybå, 2000)
Create a measurement model & test the model on
new data
(Bergersen et al., 2014)
34. Rasch model
• More than 60 years ago, Rasch (1960) used tests
to study the reading performance of students.
• Rasch models have become a practical way to
measure psychological abilities
35. Task construction
1.Picked existing tasks from literature
2.Reanalyzed data sets to combine time & quality
3.Large control group with 255 developers from 9
countries used to score tasks
36. • Collection of tasks
• VM in the cloud (Linux + IntelliJ / Eclipse
• Interchangeable
Building a new standard to measure & analyze
software engineering skills_
37. Detailed analytics of skills and
meaningful comparisons with empirically validated norm groups_
41. OCA/OCP
• Makes you think like a compiler
• Check for API knowledge
• Does not tell anything about your actual skill
level
• Doesn’t measure skills like GrepS does
• So, what if we would consult the community for
a GrepS test variant that uses both
42. Foojay community
• Over 1000 members
• 2 news articles every week day
• Podcast
• Conference information
• Tech questions
• And the certification initiative!
43.
44. Foojay certified developer
• Multiple sessions with champions and 80+
community members
• Defined what should be tested and how
• Universal test
• Never do a test again!
• Or actually do test again to prove your progress!
• Current status = beta!
• Want a free test, let me know
45. Recap
• Coding is mostly about skills
• Research shows how to measure skills
• Let’s base Java certification this!
• Questions?
Hinweis der Redaktion
GB: ..are you?
GB: drop the second «is» and add a «?»
Q & A with Bert Jan about how to test if someone is a good dev, ask audience
Hypermedia as the Engine of Application State
Single responsibility principle - Class has one job to do. Each change in requirements can be done by changing just one class.
Open/closed principle - Class is happy (open) to be used by others. Class is not happy (closed) to be changed by others.
Liskov substitution principle - Class can be replaced by any of its children. Children classes inherit parent's behaviours.
Interface segregation principle - When classes promise each other something, they should separate these promises (interfaces) into many small promises, so it's easier to understand.
Dependency inversion principle - When classes talk to each other in a very specific way, they both depend on each other to never change. Instead classes should use promises (interfaces, parents), so classes can change as long as they keep the promise.
Open Hiring: prevent ruling out certain people because of bias. Just see how they do
FAANG companies
Programming aptitude is a term that is often used to identify the reason that some developers are dispositionally better at becoming good programmers.
Thus, when new tests of programming aptitude are proposed (e.g., Dehnadi, 2006; Harris, 2014; Tukiainen & M¨onkk¨onen, 2002), the main challenge is not whether such tests predict the success in learning to program, but rather that such tests should be better than other tests that are already available for use and, furthermore, may be used in many more situations besides programming.
Would this mean that asking to explain a binary tree is not a good predictor of programming performance?
Research also shows that brain training doesn’t really work. Practising your brain by doing a lot of chess excersices doesn’t mean you become a good programmer fast. So again measure the actuall skill and not a proxy skill. (from refactoring vs refuctoering)
I used to say Bsc Informatics at a minimum. But seen too many counterexamples. It sure has a role
0,3 is very weak
Research about job performace and grades
So especially in the beginning academic background is important, later not so much. That would coincide with intuition no?
Important moderator was found to be the time between graduation and performance measurement.
Best prediction of someones future performance is past performance
Studies/papers say that in the end experience is only a limited predictor of developer performace
Key difference between senior and junior is that they’ve seen it before. This allows them to focus more on semantics rather than syntax. If you already understand an annotation in Java, you can focus on getting the job done. Before we had internet this mattered more, since it wasn’t so easy to look up syntax, but now with rich IDE’s that help us autocomplete and tools like CoPilot and ChatGPT the advantage of having it seen before becomes smaller one could argue, also according to a recent sudy from Tornhill, Borg and Moes). But learning details matter
Skilled behavior can be traced back to Aristotle (1999), who di↵erentiated between epistˆemˆe (i.e., knowledge) and technˆe (i.e., crafts, art).
The theory of skill describes three overlapping phases in the acquisition of psychomotoric skill, that is, bodily movement in relation to mental activities. In the first (cognitive) phase, facts about the domain where the developed skill will later be applied are initially acquired.
Another central predictor of skill is knowledge (Chi et al., 1988). That knowledge is central to skill follows directly from phase one of skill acquisition: Kyllonen and Woltz (1989) name this phase “knowledge acquisition”.6
Studies also show that the best way to learn something is by simulating the to be learned task as closely as possible
https://xray-delta.com/2011/10/12/dreyfus-model-a-richer-understanding-of-competency-building/
Skill acuisition via three phases (learn how to drive a car)
1. learn the facts
2. active thinking
3. embody skill
Become a car driver, learn how to play the piano
Become a chess player
Become a programmer
E’ve already seen some predictors of skill, like education and experiemce.
Level 5 skill is not enough
High performance is, in many occasions, most likely due to high skill. However, high performance may also be due to luck. Or motivation can cause bad performace, work extra hard
Context: Bidding rounds are frequently used to select competent and cost-efficient providers for software projects. Objective: We hypothesize that emphasizing low price when selecting software providers in such bidding rounds substantially increases the likelihood the project will fail. Method: The hypothesis is tested by analyzing a dataset of 4,791,067 bids for 785,326 small-scale projects registered at a web-based marketplace connecting software clients and providers. Results: We find evidence supporting our hypothesis. For example, selecting providers with bids 25% lower than the average bid is connected to a 9% increase in the frequency of project failures for the same level of provider skill. In addition, we found that clients emphasizing a low price, on average, selected providers with lower skill levels. This decrease in provider skill level further strengthened the negative effect of a strong focus on low price on project failures. For example, selecting a provider with a 15% failure rate for previous projects instead of 5% increased the failure rate by 33%. Conclusion: We interpret the findings to suggest that a client may substantially reduce the likelihood of project failure by reducing the emphasis on low price when selecting a provider.
(https://www.researchgate.net/publication/262345882_A_strong_focus_on_low_price_when_selecting_software_providers_increases_the_likelihood_of_failure_in_software_outsourcing_projects)
Selecting a provider with a 15% failure rate for previous projects instead of 5% increased the failure rate by 33%
Work focused on issues concerning the measurement of productivity (e.g., Jones, 1997; Walston & Felix, 1977) was often based on counting lines of code that are produced within a given time unit. Nevertheless, it appears that a valid measure of programming productivity has been elusive as there is no readily available way to compare the performance of individuals across di↵erent systems.
Land of the blind one eye is king
https://www.wilmarschaufeli.nl/publications/Schaufeli/358.pdf
Task performance can be defined as the proficiency (ie, competency) with which one performs central job task
Individual’s current level of programming performance on one set of tasks predicts an individual’s future level of performance on another set of tasks.
Generally, one can say that performance increases with higher expertise. But there are situations where novice programmers have been found to outperform experts (see, e.g., Adelson, 1984; Haerem & Rau, 2007). In those situations, factors other than programming expertise may dominate an individual’s performance on a task. For example, implementing a calculation may pose a complex mathematical problem rather than a programming challenge
Master of Java
Similar to the “conventional wisdom [that]‘The best indicator of future performance is past performance”’ (Wernimont & Campbell, 1968, p. 372).
Whereas performance may use units such as time, degree of correctness, efficacy, reliability, etc., none of these units are meaningful to characterize skill.
For example, for an airplane passenger, the di↵erence between landing safely (good flying performance) and dying in a crash (the worst possible flying performance) is probably infinitely large (what does this tell about the skills of a pilot)
the Rasch model conceptualizes abilities using an interval scale where the unit of measurement is the logarithm of the odds
The relation between ability and difficulty is expressed stochastically in the Rasch model, so that when the level of an individual’s ability equals the difficulty of an item, the probability of a correct answer is 50%
Time & Quality
First, The construction and sampling of tasks in the Pre Study was based on previously published tasks.
Second, to establish principles to combine time and quality as performance, two data sets were reanalyzed.
Data Set 1 was reported in (Arisholm & Sjøberg, 2004) and used only the data from the professionals in the study
Third, concerning the subject sampling and data collection, Table 4 shows all the involved subjects and data sets. In addition to Data Sets 1 and 2, another data set (Data Set 3) was used in the Main Study and is discussed in detail in Papers II, III, and IV. As shown in the table, 255 developers from 9 unique countries participated in total, representing 320 person days of programming performance available for analysis.