The document discusses transforming HTML from a third party service without a public API into a REST API. It recommends emulating a browser to issue HTTP requests and parse the resulting HTML. Tools like Wireshark, Chrome Inspector and Gargl can help analyze and record requests. The HTML can then be parsed using a library like JSoup and exposed through a framework to define API endpoints and verbs. Caching is also important to avoid overloading the third party service.
result management system report for college project
HTML to REST API thoughts
1. Cedric Gatay - c.gatay@code-troopers.com
or HTML to REST API guide
APIs for everyone
2. Cedric Gatay - c.gatay@code-troopers.com
Initial rant
• Want to build an awesome app
• On top of third party service
• No API, only HTML available
2
3. Cedric Gatay - c.gatay@code-troopers.com
Basic idea
• Emulate a browser to issue requests
• Parse resulting HTML
• Transform it into a REST API
3
4. Cedric Gatay - c.gatay@code-troopers.com
Prerequisites
• Analyze site to find endpoints
• mobile versions are great
• Reverse engineer server requests
• understand parameters might be tricky
4
6. Cedric Gatay - c.gatay@code-troopers.com
Tools
• Chrome Inspector to the rescue
6
7. Cedric Gatay - c.gatay@code-troopers.com
Tools
• Gargl : https://github.com/jodoglevy/gargl
• Record your requests
• Name them
• Parametrize them
• Generate Java/JavaScript code
7
8. Cedric Gatay - c.gatay@code-troopers.com
From HTML to API
• Gargl generates boilerplate for HTTP calls
• Refactor generated code
• Need to parse HTML
• Expose it through API
8
9. Cedric Gatay - c.gatay@code-troopers.com
Parse HTML
• JSoup : http://jsoup.org/
• simple parser
• allows to use css selectors
9
10. Cedric Gatay - c.gatay@code-troopers.com
Expose API
• Define verbs
• Use your preferred framework here
• restx : http://www.restx.io
• Test extensively
• parsing WILL break
10
11. Cedric Gatay - c.gatay@code-troopers.com
Cache API
• Do not spam base server
• Cache external calls as much as possible
• Store constant data
11