Using Headless Chrome to Find Link Redirects

Following redirects to find a final destination — seemingly easy at first. CURL the URL, let curl follow the redirects, then spit out the final destination. That was my initial thought… then the “gotcha”. We also need to follow javascript redirects to find the “final-final” destination.

Enter Headless Chrome, a PhantomJS-like tool for “viewing” web pages without a browser. The big benefit with Headless Chrome over PhantomJS being that Headless Chrome uses the latest version of the Blink rendering engine as opposed to PhantomJS’ WebKit rendering engine, the very engine that¬†Chrome ditched a while back. What this (hopefully) means for us is that when you load up a page in Headless Chrome and export it to a PDF, it should look the same as if you were to open it up in the Chrome browser. This seems like a perfect fit for our problem.

The test page

First thing’s first, we need a test page. Or, more specifically, three test pages.

On page 1, we’ll have a simple PHP redirect:

On page 2, we’ll do a JavaScript redirect:

And finally, on page 3, we’ll let ourselves know that we made it all the way to the end:

And now, just to prove that this whole process is necessary, we’ll run CURL with the -L flag to follow redirects and confirm that we’re only making it to the second page:

And if you test it in the browser, you’ll get all the way to the third page as expected.

Headless Chrome

Now let’s test it with Headless Chrome and see what we get.

First you’ll need to find your Chrome browser installation location. I’m on OSX, so it’s at

Then we’ll need to set the headless¬† flag, disable GPU acceleration, and then we’ll dump out the dom:

--headless --disable-gpu --dump-dom http://scripts.local.dev/page1.php

So all together we’ll have

Which gives us this output

Perfect! Exactly what we were expecting. Now let’s see what we have to do to get those redirects.

The Script

Now, using a slightly modified script from the chrome-remote-interface repository we can get all the redirects including the JavaScript ones:

Be sure to install the dependencies via NPM

And run the Headless Chrome instance in the background with

Run the script, and you should see something like this

And there we have our redirect chain, complete with JavaScript redirects. So now we have a script that we can plug into other parts of our system.

4 thoughts on “Using Headless Chrome to Find Link Redirects

  1. Hey Sony,

    “hops” is just an array that holds each redirect URL that we’re taken through, and that was mistakenly left out of the code. I’ve added it back in. Thanks for catching that!

  2. Nice script. Do you know how I could also get the response code of each domain redirect and then take a screen shot of the last redirect??

    For example if red.com redirects to blue.com then yellow.com. I would like to get these redirects saved to an array but also the response code (e.g 302, 302, 200) for each url too. Either saved to a different array or preferably the same one.

    Then I would like a screenshot of the last URL so in this case yellow.com.

  3. Actually upon testing this script I realise it doesn’t work.
    It only seems to get the last domain redirect URL, and also sometimes gets many other URL’s which are not redirect URL’s such as doubleclick.net and social media widgets.

    Try running the script with these examples.
    Examples:
    const initialUrl = ‘http://google.com’; (only gets last redirect)
    const initialUrl = ‘http://yahoo.com’; (gets last redirect url and also approx 20 others such as doubleclick.net and yahoo image and ad server urls)
    const initialUrl = ‘http://madonna.edu’; (gets twitter/facebook urls)
    const initialUrl = ‘http://youtube.com’; (only gets last redirect url and also doubleclick.net and ad server urls)

Leave a Reply

Your email address will not be published. Required fields are marked *