Automate web tasks like screenshots, form fills and more with Puppeteer

possibly
Puppeteer can
automate that!
with the help of minions

Önder Ceylan
@onderceylan
Sharing knowledge on #javascript, #typescript, #angular, #ionic and #pwa
JS Squad Lead @LINKIT
Speaker, Organiser @ITNEXT
Speaker, Organiser @GDG NL

Headless Chrome
chrome —-headless —-remote-debugging-port=9222

–Puppeteer docs
“Most things that you can do
manually in the browser can be
done using Puppeteer! ”

Examples to get you started
• Generate screenshots and PDFs of pages
• Crawl a SPA and generate pre-rendered
content
• Automate form submission, UI testing,
keyboard input, etc
• Capture a timeline trace of your site to
help diagnose performance issues

puppeteer.launch().then(async browser => {
  const page = await browser.newPage();
  await page.goto('https://www.linkit.nl');
  await page.screenshot({path: 'screenshot.png'});
  await browser.close();
});

github.com/transitive-bullshit/awesome-puppeteer

Alex ! "
Xander # $
Ellen % &
Yelda ' (

Emulating color scheme NEW 
in v2

await page.goto('https://googlechromelabs.github.io/dark-mode-toggle/demo/index.html'); 
await page.emulateMediaFeatures([{ name: 'prefers-color-scheme', value: 'light' }]); 
await page.screenshot({ path: 'light.jpg', type: 'jpeg', omitBackground: true }); 
await page.emulateMediaFeatures([{ name: 'prefers-color-scheme', value: 'dark' }]);
await page.screenshot({ path: 'dark.jpg', type: 'jpeg', omitBackground: true });
Emulating color scheme NEW 
in v2

await page.goto('https://frontmania.com');
await page.addScriptTag(
{ url: 'https://cdnjs.cloudflare.com/ajax/libs/axe-core/3.3.2/axe.min.js' }
);
const results = await page.evaluate(() => axe.run(document));
 
expect(results.violations.length).toBe(0);
Accessibility test

const pti = require('puppeteer-to-istanbul');
 
await page.coverage.startJSCoverage();
await page.goto('https://www.linkit.nl/');
const jsCoverage = await page.coverage.stopJSCoverage();
 
pti.write(jsCoverage);
await page.close();
Code coverage test

throttle network
track memory usage
emulate devices
run audits

github.com/ChromeDevTools/devtools-frontend

Protocol Monitor
chrome://flags/#enable-devtools-experiments

chrome —-remote-debugging-port=9222
Chrome DevTools
Protocol

chromedevtools.github.io/devtools-protocol

github.com/ChromeDevTools/awesome-chrome-devtools

Using DevTools
Protocol with PPTR

puppeteer.launch().then(async browser => {
const session = await page.target().createCDPSession();
  session.on('Animation.animationCreated', () => {
    console.log('Animation created!');
  });
  await session.send('Animation.enable');
  await session.send('Animation.setPlaybackRate', {
    playbackRate: 10,
  });
});

Puppeteer Core
npm i puppeteer-core

Puppeteer Core
browser = await puppeteer.connect({
browserWSEndpoint: 'ws://127.0.0.1:9222/.../...'
});
// or
browser = await puppeteer.launch({
executablePath: '/Applications/Google Chrome.app
/Contents/MacOS/Google Chrome'
});

Chrome Launcher
npm i chrome-launcher

// Launches a debugging instance of Chrome
function launchChrome(headless=true) {
  return chromeLauncher.launch({
    // port: 9222,
    chromeFlags: [
      '--window-size=412,732',
      '--disable-gpu',
      headless ? '--headless' : ''
    ]
  });
}
launchChrome().then(chrome => {
  console.log(`Chrome debuggable on port: ${chrome.port}`);
  // chrome.kill();
});
chrome instance

Puppeteer Light
npm i puppeteer-core
npm i chrome-launcher

github.com/onderceylan/pwa-asset-generator

await Promise.all(filters.map(async (filter) => {
await page.setContent(`
<style>/* Page styles here */</style>
<figure class="${filter}">
<img src="${getImageBase64Url('./sample.jpg')}">
<span>@onderceylan</span>
</figure>
`, { waitUntil: 'networkidle2' });
await page.addStyleTag({
url: 'https://cdnjs.cloudflare.com/ajax/libs/cssgram/0.1.10/cssgram.min.css'
});
// Get original image dimensions
const { width, height } = await page.evaluate(() => {
return (({naturalWidth: width, naturalHeight: height}) =>
({width, height}))(document.querySelector('img'));
});
await page.setViewport({ width, height });
await page.screenshot({
path: `pptgram/pptrgram-${filter}.jpeg`,
type: 'jpeg',
quality: 70,
fullPage: true,
});
});
PptrGram

await page.goto(`http://localhost:8002/tensorflow.html`);
const result = await page.evaluate(() => {
const img = document.getElementById('img');
// Load the model
return cocoSsd.load().then(model => model.detect(img))
});
console.log(result);
PptrFlow

await page.goto('https://www.linkit.nl/');
expect(await page.content()).toMatchSnapshot();
DOM Snapshot Testing

await page.tracing.start({ path: 'trace.json' }); 
await page.goto('https://www.google.com'); 
await page.tracing.stop();
Timeline Trace Monitoring

const protocol = await page.target().createCDPSession();
await protocol.send('HeapProfiler.enable');
await protocol.send('HeapProfiler.collectGarbage');
const startMetrics = await page.metrics();
// Do memory regressions here by interacting with the page
await protocol.send('Input.synthesizeScrollGesture', {
x: 100,
y: 100,
yDistance: -400,
repeatCount: 3
});
await protocol.send('HeapProfiler.collectGarbage');
const endMetrics = await page.metrics();
expect(endMetrics.JSHeapUsedSize < startMetrics.JSHeapUsedSize * 1.1)
.toBeTruthy();
Memory leak by Heap

// Get a handle to the Map object prototype
const mapPrototype = await page.evaluateHandle(() => Map.prototype);
// Query all map instances into an array
const mapInstances = await page.queryObjects(mapPrototype);
// Count amount of map objects in heap
const count = await page.evaluate(maps => maps.length, mapInstances);
// Idea here is to test object instances on the page
// where it's expected to be invalidated
expect(count).toBe(0);
Memory leak by Prototype

Monitor change on security state

const protocol = await page.target().createCDPSession(); 
await protocol.send('Security.enable');
protocol.on('Security.securityStateChanged', console.log);
Monitor change on security state

Monitor SSL certificate expiration

page.on('response', (resp) => {
const url = resp.url(); 
if (url === siteUrl) {
const secDetails = resp.securityDetails();
const now = Math.floor((new Date()).getTime() / 1000); 
console.log((Math.floor((secDetails.validTo() - now) / 86400)),
'days to expire');
}
});
await page.goto(siteUrl, { waitUntil: 'networkidle0' });
Monitor SSL certificate expiration

Main Takeaways
• Chrome can be instrumented with a WS connection over
Chrome DevTools Protocol — CDP
• You might not need to download chromium revision every time
—puppeteer-core + chrome-launcher
• Headless chrome can be executed on servers— cloud, and CI
pipelines
• You can automate anything you do on DevTools, by using raw
protocol of CDP on puppeteer—CDPSession

Thank you!
@onderceylan
https://github.com/onderceylan/puppeteer-demos

Automate web tasks like screenshots, form fills and more with Puppeteer

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Automate web tasks like screenshots, form fills and more with Puppeteer

Ähnlich wie Automate web tasks like screenshots, form fills and more with Puppeteer (20)

Mehr von Önder Ceylan

Mehr von Önder Ceylan (6)

Kürzlich hochgeladen

Kürzlich hochgeladen (11)

Automate web tasks like screenshots, form fills and more with Puppeteer