The low-code/no-code movement has the core aim of making technical tools more accessible to people without a technical background. For example, there are multiple tools that allow people to create applications and to test their functionalities without writing a single line of code. What currently is hard to find on the market is the possibility of testing and monitoring data quality without requiring coding skills. Data quality has a significant impact on the training of AI models and on the correct functioning of web and mobile applications. However, testing data quality is a burdensome task as it requires both technical and domain expertise. No-code data quality testing could allow domain experts to promptly catch issues and fix them, however there are multiple challenges that have to be taken into account to make this concept become a reality. What are the key challenges and how could we overcome them? We will discuss about it during the talk.

1. How can we make
no-code data quality
testing become reality?
Challenges of no-code data quality testing
and possible solutions

3. we love data

4. we love high quality data

5. Wait, haven’t I already
said that?

6. us 😊

7. What is high quality data
and its characteristics

8. How can we create
codeless tests
for data quality

9. data quality

10. Data quality testing concept
ODH day 2022

11. Data quality testing concept
ODH day 2022
Development of codeless data quality testing

12. Data quality testing concept
ODH day 2022
Development of codeless data quality testing

13. What are the challenges for
no-code data quality testing?

14. We want that non-technical users
are able to analyse the data quality
of domain-independent JSON

15. We want that non-technical users
are able to analyse the data quality
of domain-independent JSON

16. We want that non-technical users
are able to analyse the data quality
of domain-independent JSON

17. Data quality rules
Data quality rules
validity accuracy
completeness
integrity consistency
timeless

18. We want that non-technical users
are able to analyse the data quality
of domain-independent JSON

19. Key constraints that cannot
be overlooked for the development
of no-code data quality tests

20. Non-technical users
- Lack of knowledge
- Usability challenges

21. Data quality
- “Right or wrong” highly dependent on intended use of
dataset
- Semantic understanding of data is needed, but not
enough data to train AI for this specific purpose

22. Domain-independent JSON
- Variable length and depth
- No predefined structure and fields

23. How have we overcome
these challenges?

24. Insertion of url
of API endpoint

25. There is a tutorial that
explains data quality
concepts applied to the
inserted dataset.

26. Decomposition of
JSON dataset in
elements and types

27. Suggestion of possible
data quality issues.
The user approves or
discards them

28. The user can add also
other issues that were
not identified

29. The selected fields
are monitored based
on the information
inserted by the user

30. The tech challenges
behind it

31. How to perform
dynamic JSON
analysis?

32. We cannot make any assumption
of what we will receive, any
operation should before check the
expected value

33. Handle authentication, content
type and http response code

34. JSON that we process can be of
any structure (object, array, of object …)

35. JSON is a tree and can be
traversed using a recursive
visit algorithm

36. Json library allow to detect the
type of an attribute at runtime

37. How to suggest
automatically
possible data quality
issues?

38. Detection of mixed data types as it
indicates possible data validity issues
“attr” : 0,1,’a’,3,4

39. Detection of null/undefined attributes as it
indicates possible data completeness issues
“attr”: null or missing

40. Detection of outliers as it indicates
possible data validity issues
“attr”: outside range?

41. Detection of abnormal standard deviation
as it could indicate possible data accuracy
issues
sd = 0 ? ☠ ?

42. Possible implications

43. Security implication:
Protect services from malicious
users (intentionally or not)

44. Privacy implication:
Don’t store sensitive data (i.e.
forget json data after analysis)

45. Conclusions

46. Codeless data quality
testing
Usability and technical
challenges
Our solution

47. Are you ready to jump in the
world of codeless data quality
testing?

48. Become an early adopter

49. CatchSolve info@catch-solve.tech
www.catch-solve.tech