Headless browsers allow testing websites without a graphical user interface, improving test speed. Popular options include Headless Chrome, which provides the same capabilities as Chrome but without GUI overhead. Puppeteer is a Node library that controls Headless Chrome, enabling tasks like taking screenshots, testing JavaScript, and simulating user interactions at a faster, more optimized pace than alternatives like Selenium. Headless browsers and tools like Puppeteer and Chrome DevTools help speed up the development process through quicker, more efficient testing.

1. Headless Browser - A Stepping Stone
Towards Developing Smarter Web
Applications
By prerna.bhatt
December 28, 2020
11 min Read
Share This Article
Websites being the primary source of communication in the digital transformation world, have evolved humongously since the last decade. Web
development has grown at a tremendous pace with lots of automation testing frameworks coming in for both frontend and backend development.
Websites have become smarter and so have the underlying tools and frameworks. With a significant surge in the web development area, the browsers
have also become smarter. Nowadays, you can find headless browsers, where users can interact with the browser without GUI. You can even scrape
websites in headless browsers using packages like Puppeteer and nodejs.
Efficient web development hugely relies on a testing mechanism for quality assessment before we can push them in production environments. Headless
browsers can perform end to end testing, smoke testing etc at a faster speed as it is free from overhead of the memory space required for the UI.
Moreover, studies have proved that the headless browsers generate more traffic than the non-automated ones. Popular browsers like Chrome can even
help in debugging the web pages in real time, analyse performance, notify the memory consumption, enable tweaking of the code and analyse
performance in real time etc. Isn’t this evolution of browsers heading towards a smarter web development process? So in this blog we will have an
overview on headless browsers and understand how it helps in smarter and faster website development.
What is a Headless Browser ?
A headless browser is simply a browser without the GUI. It has got all the capabilities of rendering a website, like a normal website. Since GUI is not
available in the browser with headless mode, we need to use the command line to interact with the browser. Headless browsers are designed for tasks
like automation testing, javascript library testing, javascript simulation and interactions.
One of the biggest reasons for using headless browser or headless browser testing is that it let’s you run the tests more quickly and in a real
environment. For eg, the combination of chrome devtools and headless chrome lets you edit pages on the fly and, which helps you in diagnosing the
problem quickly ultimately helping you in developing better websites faster. So headless browsers are more fast, flexible and optimised in performing
tasks like web-based automation testing. Like a normal browser, the headless browser is capable of performing tasks like links parsing JavaScript,
clicking on links, coping with any downloads and for executing this we need to use command line. So it can provide a real browser context without any
of the memory consumed for running a full-fledged browser without a GUI.

2. The Need for a Headless Browser
With advancements in the website development technologies, website testing has taken a center stage and emerged as the most essential steps in
developing high performing websites. Even browsers are becoming smarter as they can load the javascript libraries for performing automation testing.
Isn’t that a significant transformational leap in the website testing. So let’s have an overview on some of the major functions performed by headless
browsers.
Enables faster web testing using Command Line Interface
With headless cross browser testing, we are saved from the overhead of memory consumed in GUI, hence it enables faster website testing, using
command line as the primary source of interaction. The headless browsers are designed to execute crucial test cases like end to end testing which
ensures that the flow of an application is performing as designed from start to finish. The headless browsers cater to this use case as they enable faster
website testing.
Scraping websites
The headless browser saves the overhead of opening the GUI thus enabling faster scraping of the websites. In headless browsers we can automate the
scraping mechanism and extract the data in a much more optimised manner.
Taking web screenshots
Though the headless browsers do not avail any GUI, they do allow the users to take snapshots of the website that they are rendering. It’s very useful in
cases where the tester is testing the website and needs to visualise the code effects and save results in the form of screenshots. In a headless browser
you can easily take a large number of screenshots without any actual UI.
Mapping user journey across the websites
Headless browsers allow you to programmatically map the customer journey test cases. Here headless browsers help the users to optimise the user
experience throughout their decision making journey on the website.
Now that we have understood what is a headless browser and it’s numerous features, along with the its key quality of being a lightweight browser which
helps in accelerating the speed of testing, let’s have an overview on the most popular headless browser, Headless Chrome and see what does it
unlocks
Diving into Headless Chrome and Chrome DevTools
We have a number of headless browsers and to name a few are firefox version 55 and 56, PhantomJs, Html Unit, Splinter, jBrowserDriver etc. Chrome
59 is Chrome’s version to run it in headless mode. Headless Chrome and Chrome DevTools is quite a powerful combination enabling the users with out
of box features. So let’s have an overview on Headless Chrome and Chrome DevTools.
What is Headless Chrome

3. Headless Chrome shipping in Chrome 59, is basically running chrome in a headless environment. It’s running chrome without GUI. This light weighted,
memory sparing and quick running browser brings in all the modern web platform features provided by the chromium and blink rendering engine to the
command line.
As per the studies, the automated browsers always generated more traffic than the non-automated ones. In a recent survey, it was discovered that the
headless Chrome generated more traffic than its previous leader Phantum Js within a year of its release.
Apart from this, there are several reasons why chrome is the most popular headless browser. One of the reasons is, it’s always updating out of the
box features, which constantly introduce new trends in web development. It also consists of a rendering engine called Blink , which constantly
updates itself as the website evolves. What does the Headless Chrome Unlocks :
The ability to test the latest web platform features like ES6 modules, service worker and
streams
Enables to programmatically tap into the chrome devtools and makes use of the awesome
features like network throttling, device emulating, website performance analysis etc.
Test multiple levels of navigation
Gather page information
Take screenshots
Create PDFs
Now let’s have a look on the most common flags to start working with headless chrome:
Starting headless
For starting headless, you need a Chrome 59+ and open the chrome binary from the command line. If you have got the Chrome 59+ installed then start
the Chrome with
The — headless flag starts the chrome in headless mode

4. Similarly for printing the DOM, creating pdf, taking screenshots we can simply use the following flags
Printing the DOM
The –dump-dom flag prints document.body.innerHTML to stdout
Create a PDF
The –print-to-pdf flag creates a PDF of the page:
Taking screenshots
To capture a screenshot of a page, use the –screenshot flag
Debugging a code without the browser UI
If you want to debug your code in a headless browser using the chrome devtools then make note of the following flag.–remote-debugging-
port=9222. This flag helps you to open the headless chrome in a special mode, wherein the chrome devtool can interact with the headless browser to
edit the web page during run-time. We will dig deeper into Chrome Devtools in the later section of the blog.
For debugging the web page with chrome devtools, use the –remote-debugging-port=9222 flag.
What is Chrome DevTool ?
Chrome DevTools is a set of web developer tools built directly into Google Chrome. It helps to debug the web pages on the fly and hence helps to detect
the bugs quickly, which ultimately helps to develop the websites faster.
The simplest way of opening devtools is, right click on your webpage and click inspect. Now based on your purpose to use the devtool, you can open
various consoles. For example to work with DOM or CSS you can click on the elements panel, to see logged messages or run javascript click on
the console panel, for debugging the javascript click on the source panel, to view network activity click on the network panel, to analyse the
performance of the webpage click on the performance panel, to fix memory problems click on the memory panel.
As we can see, the chrome devtools is a package of diverse functionalities which helps in debugging a web page in the chrome browser. But what about
the headless browser with no UI, how can we debug the web page with no UI?. Can chrome devtools help debugging a headless browser ? Let’s
demystify the ways of debugging a headless browser with chrome devtools, discuss what is puppeteer in the following sections of the blog.
Puppeteer
As discussed earlier, one of the ways of debugging a web page in a headless browser is to use the flag –remote-debugging-port=9222 in the
command line which helps in tapping into the chrome devtools programmatically. But, there is another layer into the picture to play with the headless
chrome to perform numerous out of the box tasks and make use of headless in a more efficient way. Here, Puppeteer comes into the picture.

5. Puppeteer is a Node Library which provides a high level API to control chrome over devtools protocol. Puppeteer is usually headless but can also be
configured to use full non – headless Chrome. It provides full access to all the features of headless chrome and can also run chrome fully in a remote
server which is very beneficial to the automation teams. It would be quite justifying to term Puppeteer as Google Chrome team’s official Chrome
headless browser.
One of the greatest advantages of using puppeteer as an automation framework for testing is that unlike other frameworks, it is very simple and easy to
install.
As puppeteer is a node javascript library, so first of all you need to install nodejs on your system. Nodejs comes with the npm (node package manager)
which will help us to install the puppeteer package.
The following code snippet will help you to install nodejs
## Updating the system libraries
##sudo apt-get update
## Installing node js in the system
##sudo apt-get install nodejs
Once you are done with installation of node js in your machine, you can run the following flag to install puppeteer.
npm i puppeteer
With this you completed the installation for puppeteer which will also by default download the latest version of chrome.
Why is puppeteer so useful ?
Puppeteer provides full access to all the out of box features provided by the headless chrome and its constantly updating rendering engine called blink.
Other than mostly used automation testing frameworks for web applications like selenium web driver, puppeteer is so popular as it provides automation
for a light weighted (UI less) headless browser which helps in testing faster. Likewise there are multiple functionalities provided by puppeteer, so let’s
have a look at it
It can help generate screenshots and pdfs of pages
Crawl a single page application and generate pre-rendered content
Automate form submission, UI testing, end to end testing, smoke testing, keyboard input etc
Creates an up-to-date, automated testing environment. Run your tests directly in the latest
version of Chrome using the latest JavaScript and browser features.
Captures a timeline trace of your site and analyses performance issues.
It can test Chrome Extensions.
Allows performing web scraping
As we are aware of the functionalities performed by puppeteer let’s have an overview on the code snippet for taking screenshot in puppeteer.
const puppeteer = require(‘puppeteer’);
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto(‘https://example.com’);

6. await page.screenshot({path: ‘example.png’});
await browser.close();
})();
Once this code gets executed, a screenshot will be saved in your system through the path mentioned in the code snippet.
Conclusion
Faster and qualitative web development has always been and will always be the top priority of QA and development teams. The headless browsers (
without GUI) being light weighted and memory sparing runs at a higher speed while automation testing. Definitely they cater to the need of smarter web
development. Moreover, they help in testing all the modern web platform features as well as enable debugging and performance analysis during real
time which adds another feather on the cap. They are responsible for heavy traffic in the web applications and support website scraping with the help of
packages like nodejs and puppeteer. Furthermore, the installation of headless browsers is easier than installation of any other web automation
frameworks like selenium. These qualities of headless browsers are compelling the brands to utilize them for developing high performance web
applications.