SlideShare a Scribd company logo
1 of 3
Download to read offline
Lesson 19. Pattern 11. Serialization and
data interchange
Succession to existing data interchange protocols is an important component of the process of porting a
program solution to a new platform. You need to provide the capability of reading the existing projects'
formats, data interchange between 32-bit and 64-bit processes, etc.

Basically, the errors of this pattern concern serialization of memsize-types and data interchange
operations they are used in:

1) size_t PixelsCount;

    fread(&PixelsCount, sizeof(PixelsCount), 1, inFile);

2) __int32 value_1;

    SSIZE_T value_2;

    inputStream >> value_1 >> value_2;

These samples contain errors of two kinds: using types of changeable capacity in binary interfaces and
ignoring the byte order.


Using types of changeable capacity
Do not use types that change their sizes depending upon the development environment in binary
interfaces of data interchange. All the types in C++ do not have a fixed size, so they cannot be used for
this purpose. That is why developers of software development tools and programmers themselves
create data types that have a strictly fixed size such as __int8, __int16, INT32, word64, etc.

These types enable data interchange between programs on various platforms although it requires some
additional efforts. The two examples shown above are written incorrectly and it will become clear when
some data types change their sizes from 32 bits to 64 bits. Keeping in mind the necessity of supporting
obsolete data types, you may correct these samples in the following way:

1) size_t PixelsCount;

    __uint32 tmp;

    fread(&tmp, sizeof(tmp), 1, inFile);

    PixelsCount = static_cast<size_t>(tmp);

2) __int32 value_1;

    __int32 value_2;

    inputStream >> value_1 >> value_2;

But this way of correcting the code is not the best. The program may process more data after being
ported to a 64-bit system, so 32-bit types in the code may become a great obstacle. In this case you may
leave the obsolete code as it is to make it compatible with the obsolete data format but fix the incorrect
types. Then you may create a new binary data format taking into consideration the previous errors. One
more way out is to refuse to use binary formats and take the text format or other formats provided by
various libraries.


Ignoring the byte order
Even after solving the issue of type sizes, you may encounter the problem of incompatibility of binary
formats. The cause lies in a different data representation. It is most often related to a different byte
order.

A byte order is a method of writing the bytes of multibyte numbers (see also Figure 1). The little-endian
byte order means that the writing begins with the least-significant byte and ends with the most-
significant byte. This writing order is employed in the memory of personal computers with x86 and x86-
64-processors. The big-endian byte order means that the writing begins with the most-significant byte
and ends with the least-significant byte. This order is a standard for TCP/IP protocols. That is why the
big-endian byte order is often called the network byte order. This byte order is used in Motorola 68000
and SPARC processors.




             Figure 1 - The byte order in a 64-bit type in little-endian and big-endian systems

While developing a binary interface or data format, you should remember about the byte order. If the
64-bit system you are porting your 32-bit application to has a byte order different from that of your
application, you will have to adjust your code to this difference. To convert the big-endian byte order
into the little-endian byte order and vice versa you may use the functions htonl(), htons(), bswap_64,
etc.

Note. Many systems lack functions like bswap_64 while the function ntohl() allows you to reverse only
32-bit values. They forgot to add a version of this function for 64-bit types for some reason. If you need
to change the byte order in a 64-bit variable, see the discussion of the topic "64 bit ntohl() in C++ ?" on
stackoverflow.com site - there are several examples of how to implement this function.
Diagnosis
Unfortunately, PVS-Studio does not provide diagnosis for this pattern of 64-bit errors because this
process cannot be formalized (we cannot compose a diagnostic rule). The only thing we may
recommend you is to look through all the code fragments that are responsible for writing and reading
data as well as sending data into other processes through the COM technology, for instance.

We would be glad if somebody of our readers proposed some ideas of how to detect the errors of this
kind, at least partly.

The course authors: Andrey Karpov (karpov@viva64.com), Evgeniy Ryzhkov (evg@viva64.com).

The rightholder of the course "Lessons on development of 64-bit C/C++ applications" is OOO "Program
Verification Systems". The company develops software in the sphere of source program code analysis.
The company's site: http://www.viva64.com.

Contacts: e-mail: support@viva64.com, Tula, 300027, PO box 1800.

More Related Content

Viewers also liked

Viewers also liked (9)

Analysis of the Ultimate Toolbox project
Analysis of the Ultimate Toolbox projectAnalysis of the Ultimate Toolbox project
Analysis of the Ultimate Toolbox project
 
Testing parallel programs
Testing parallel programsTesting parallel programs
Testing parallel programs
 
Introduction into the problems of developing parallel programs
Introduction into the problems of developing parallel programsIntroduction into the problems of developing parallel programs
Introduction into the problems of developing parallel programs
 
Optimization in the world of 64-bit errors
Optimization  in the world of 64-bit errorsOptimization  in the world of 64-bit errors
Optimization in the world of 64-bit errors
 
Lesson 16. Pattern 8. Memsize-types in unions
Lesson 16. Pattern 8. Memsize-types in unionsLesson 16. Pattern 8. Memsize-types in unions
Lesson 16. Pattern 8. Memsize-types in unions
 
Lesson 21. Pattern 13. Data alignment
Lesson 21. Pattern 13. Data alignmentLesson 21. Pattern 13. Data alignment
Lesson 21. Pattern 13. Data alignment
 
VivaMP - a tool for OpenMP
VivaMP - a tool for OpenMPVivaMP - a tool for OpenMP
VivaMP - a tool for OpenMP
 
Parallel programs to multi-processor computers!
Parallel programs to multi-processor computers!Parallel programs to multi-processor computers!
Parallel programs to multi-processor computers!
 
The use of the code analysis library OpenC++: modifications, improvements, er...
The use of the code analysis library OpenC++: modifications, improvements, er...The use of the code analysis library OpenC++: modifications, improvements, er...
The use of the code analysis library OpenC++: modifications, improvements, er...
 

Similar to Lesson 19. Pattern 11. Serialization and data interchange

Similar to Lesson 19. Pattern 11. Serialization and data interchange (20)

Comparison of analyzers' diagnostic possibilities at checking 64-bit code
Comparison of analyzers' diagnostic possibilities at checking 64-bit codeComparison of analyzers' diagnostic possibilities at checking 64-bit code
Comparison of analyzers' diagnostic possibilities at checking 64-bit code
 
Static code analysis for verification of the 64-bit applications
Static code analysis for verification of the 64-bit applicationsStatic code analysis for verification of the 64-bit applications
Static code analysis for verification of the 64-bit applications
 
20 issues of porting C++ code on the 64-bit platform
20 issues of porting C++ code on the 64-bit platform20 issues of porting C++ code on the 64-bit platform
20 issues of porting C++ code on the 64-bit platform
 
Optimization of 64-bit programs
Optimization of 64-bit programsOptimization of 64-bit programs
Optimization of 64-bit programs
 
Safety of 64-bit code
Safety of 64-bit codeSafety of 64-bit code
Safety of 64-bit code
 
Driver Development for Windows 64-bit
Driver Development for Windows 64-bitDriver Development for Windows 64-bit
Driver Development for Windows 64-bit
 
A collection of examples of 64 bit errors in real programs
A collection of examples of 64 bit errors in real programsA collection of examples of 64 bit errors in real programs
A collection of examples of 64 bit errors in real programs
 
The static code analysis rules for diagnosing potentially unsafe construction...
The static code analysis rules for diagnosing potentially unsafe construction...The static code analysis rules for diagnosing potentially unsafe construction...
The static code analysis rules for diagnosing potentially unsafe construction...
 
A Collection of Examples of 64-bit Errors in Real Programs
A Collection of Examples of 64-bit Errors in Real ProgramsA Collection of Examples of 64-bit Errors in Real Programs
A Collection of Examples of 64-bit Errors in Real Programs
 
Seven Steps of Migrating a Program to a 64-bit System
Seven Steps of Migrating a Program to a 64-bit SystemSeven Steps of Migrating a Program to a 64-bit System
Seven Steps of Migrating a Program to a 64-bit System
 
Seven Steps of Migrating a Program to a 64-bit System
Seven Steps of Migrating a Program to a 64-bit SystemSeven Steps of Migrating a Program to a 64-bit System
Seven Steps of Migrating a Program to a 64-bit System
 
A Collection of Examples of 64-bit Errors in Real Programs
A Collection of Examples of 64-bit Errors in Real ProgramsA Collection of Examples of 64-bit Errors in Real Programs
A Collection of Examples of 64-bit Errors in Real Programs
 
Lesson 26. Optimization of 64-bit programs
Lesson 26. Optimization of 64-bit programsLesson 26. Optimization of 64-bit programs
Lesson 26. Optimization of 64-bit programs
 
64 bits for developers
64 bits for developers64 bits for developers
64 bits for developers
 
PVS-Studio, a solution for developers of modern resource-intensive applications
PVS-Studio, a solution for developers of modern resource-intensive applicationsPVS-Studio, a solution for developers of modern resource-intensive applications
PVS-Studio, a solution for developers of modern resource-intensive applications
 
The reasons why 64-bit programs require more stack memory
The reasons why 64-bit programs require more stack memoryThe reasons why 64-bit programs require more stack memory
The reasons why 64-bit programs require more stack memory
 
Operating system
Operating systemOperating system
Operating system
 
What is-32-bit-and-64-bit
What is-32-bit-and-64-bitWhat is-32-bit-and-64-bit
What is-32-bit-and-64-bit
 
C++11 and 64-bit Issues
C++11 and 64-bit IssuesC++11 and 64-bit Issues
C++11 and 64-bit Issues
 
Cpu 64x architecture
Cpu 64x architectureCpu 64x architecture
Cpu 64x architecture
 

Recently uploaded

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 

Recently uploaded (20)

Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024Manulife - Insurer Innovation Award 2024
Manulife - Insurer Innovation Award 2024
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 

Lesson 19. Pattern 11. Serialization and data interchange

  • 1. Lesson 19. Pattern 11. Serialization and data interchange Succession to existing data interchange protocols is an important component of the process of porting a program solution to a new platform. You need to provide the capability of reading the existing projects' formats, data interchange between 32-bit and 64-bit processes, etc. Basically, the errors of this pattern concern serialization of memsize-types and data interchange operations they are used in: 1) size_t PixelsCount; fread(&PixelsCount, sizeof(PixelsCount), 1, inFile); 2) __int32 value_1; SSIZE_T value_2; inputStream >> value_1 >> value_2; These samples contain errors of two kinds: using types of changeable capacity in binary interfaces and ignoring the byte order. Using types of changeable capacity Do not use types that change their sizes depending upon the development environment in binary interfaces of data interchange. All the types in C++ do not have a fixed size, so they cannot be used for this purpose. That is why developers of software development tools and programmers themselves create data types that have a strictly fixed size such as __int8, __int16, INT32, word64, etc. These types enable data interchange between programs on various platforms although it requires some additional efforts. The two examples shown above are written incorrectly and it will become clear when some data types change their sizes from 32 bits to 64 bits. Keeping in mind the necessity of supporting obsolete data types, you may correct these samples in the following way: 1) size_t PixelsCount; __uint32 tmp; fread(&tmp, sizeof(tmp), 1, inFile); PixelsCount = static_cast<size_t>(tmp); 2) __int32 value_1; __int32 value_2; inputStream >> value_1 >> value_2; But this way of correcting the code is not the best. The program may process more data after being ported to a 64-bit system, so 32-bit types in the code may become a great obstacle. In this case you may
  • 2. leave the obsolete code as it is to make it compatible with the obsolete data format but fix the incorrect types. Then you may create a new binary data format taking into consideration the previous errors. One more way out is to refuse to use binary formats and take the text format or other formats provided by various libraries. Ignoring the byte order Even after solving the issue of type sizes, you may encounter the problem of incompatibility of binary formats. The cause lies in a different data representation. It is most often related to a different byte order. A byte order is a method of writing the bytes of multibyte numbers (see also Figure 1). The little-endian byte order means that the writing begins with the least-significant byte and ends with the most- significant byte. This writing order is employed in the memory of personal computers with x86 and x86- 64-processors. The big-endian byte order means that the writing begins with the most-significant byte and ends with the least-significant byte. This order is a standard for TCP/IP protocols. That is why the big-endian byte order is often called the network byte order. This byte order is used in Motorola 68000 and SPARC processors. Figure 1 - The byte order in a 64-bit type in little-endian and big-endian systems While developing a binary interface or data format, you should remember about the byte order. If the 64-bit system you are porting your 32-bit application to has a byte order different from that of your application, you will have to adjust your code to this difference. To convert the big-endian byte order into the little-endian byte order and vice versa you may use the functions htonl(), htons(), bswap_64, etc. Note. Many systems lack functions like bswap_64 while the function ntohl() allows you to reverse only 32-bit values. They forgot to add a version of this function for 64-bit types for some reason. If you need to change the byte order in a 64-bit variable, see the discussion of the topic "64 bit ntohl() in C++ ?" on stackoverflow.com site - there are several examples of how to implement this function.
  • 3. Diagnosis Unfortunately, PVS-Studio does not provide diagnosis for this pattern of 64-bit errors because this process cannot be formalized (we cannot compose a diagnostic rule). The only thing we may recommend you is to look through all the code fragments that are responsible for writing and reading data as well as sending data into other processes through the COM technology, for instance. We would be glad if somebody of our readers proposed some ideas of how to detect the errors of this kind, at least partly. The course authors: Andrey Karpov (karpov@viva64.com), Evgeniy Ryzhkov (evg@viva64.com). The rightholder of the course "Lessons on development of 64-bit C/C++ applications" is OOO "Program Verification Systems". The company develops software in the sphere of source program code analysis. The company's site: http://www.viva64.com. Contacts: e-mail: support@viva64.com, Tula, 300027, PO box 1800.