Automated Load Testing

My previous post covered finding a random point within a bounds, which worked great for testing random locations throughout the country for an app. Then we realized that we did not have a way to reliably load test our API endpoint with random locations.

After some googling and some scripting, I ended up with something that worked pretty well.

There are a couple of requirements we need to meet:

1. Make it throttle-able
2. Make it trackable
3. Make it random

In the past I’ve used Apache load balancer, but unfortunately this won’t work for us because 1) we’re going to be doing a POST request and 2) the data is going to be random.

I’ll save the hour of trial/error to get it working with my requirements, and just post the final script:

max="$1"
date
# optional
echo "url: https://mywebsite.local/setuserlocation
rate: $max calls / second"
START=$(date +%s);

get () {
	curlscript=$(php loadtest.php)
  	echo $curlscript 2>&1 | { bash -C; echo $curlscript; } 2>&1 | awk -v date="$(date +'%r')" '{print $0"\n-----", date}' >> perf-test.log
}

while true
do
  echo $(($(date +%s) - START)) | awk '{print int($1/60)":"int($1%60)}'
  sleep 1

  for i in `seq 1 $max`
  do
    get &
  done
done

This will execute a PHP script (similar to the script we created in the previous post which I will post below) which outputs a CURL command X number of times per second, and outputs the result into a log file. This works by spawning our script each time in a new thread. Run it with:

sh ./loadtest.sh 10

The output will look something like:

                                 Dload  Upload   Total   Spent    Left  Speed
----- 08:08:41 PM
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
100    60  100    14  100    46      8     26  0:00:01  0:00:01 --:--:--    26
----- 08:08:41 PM
{"locationdata":"{}"}curl --header "Content-Type: application/json" --request POST --data '{"latitude":31.411381,"longitude":-111.159139}' https://mywebsite.local/setuserlocation

Here is our slightly modified PHP file. This adds a new function to create a CURL string and then echo it.

// define bounding points
// "cardinal" key not required, but useful for
// keeping track of the points
$points = [
	[
		"cardinal" => "NW",
		"Y" => 48.098490,
		"X" => -124.420681
	],
	[
		"cardinal" => "SW",
		"Y" => 32.469436,
		"X" => -117.482643
	],
	[
		"cardinal" => "SE",
		"Y" => 25.121507,
		"X" => -81.038997
	],
	[
		"cardinal" => "NE",
		"Y" => 47.162174,
		"X" => -68.432636
	]
];

$randomPosition = getRandomCoordinateWithinBounds($points);
echo getCurlCommandFromPosition($randomPosition);

/**
 * Format a string into a curl command with our given X/Y coordinate
 * @param  array $position  a given point within our testing area
 * @return string           a string formatted curl command
 */
function getCurlCommandFromPosition($position)
{
	return 'curl --header "Content-Type: application/json" --request POST --data \'' . json_encode(["latitude" => $position['Y'], "longitude" => $position['X']]) . '\' https://mywebsite.local/setuserlocation';
}

/**
 * Find the bounds of all given points and return a point within
 * those bounds
 * @param  array $points an array of X/Y points
 * @return array         an array containing a random point
 *                       within the given bounds
 */
function getRandomCoordinateWithinBounds($points)
{
	$minx = 0;
	$maxx = 0;
	$miny = 0;
	$maxy = 0;

	// find the minimum and maximum X and Y points
	// across all given points
	for ($i = 0; $i < count($points); $i++) {
		if ($points[$i]["X"] > $maxx) {
			$maxx = $points[$i]["X"];
		}
		if ($points[$i]["X"] < $minx) {
			$minx = $points[$i]["X"];
		}
		if ($points[$i]["Y"] > $maxy) {
			$maxy = $points[$i]["Y"];
		}
		if ($points[$i]["Y"] < $minx) {
			$miny = $points[$i]["Y"];
		}
	}

	// create a random point within our minX/maxX and minY/maxY bounds
	$randomPoint = [
		"Y" => round($miny + ($maxy - $miny) * (mt_rand() / mt_getrandmax()), 6),
		"X" => round($minx + ($maxx - $minx) * (mt_rand() / mt_getrandmax()), 6)
	];

	// check if the random point is within our bounds
	// if not, create a new random point and try again
	while(!pointIsInCoordinates($points, $randomPoint)) {
		$randomPoint = [
			"Y" => round($miny + ($maxy - $miny) * (mt_rand() / mt_getrandmax()), 6),
			"X" => round($minx + ($maxx - $minx) * (mt_rand() / mt_getrandmax()), 6)
		];
	}

	return $randomPoint;
}

function pointIsInCoordinates($points, $point)
{
	// get the point at the end of the array
	$j = count($points) - 1;
	$output = false;

	for ($i = 0; $i < count($points); $i++)
	{
	    if ($points[$i]["X"] < $point["X"] && $points[$j]["X"] >= $point["X"] || $points[$j]["X"] < $point["X"] && $points[$i]["X"] >= $point["X"])
	    {
	        if ($points[$i]["Y"] + ($point["X"] - $points[$i]["X"]) / ($points[$j]["X"] - $points[$i]["X"]) * ($points[$j]["Y"] - $points[$i]["Y"]) < $point["Y"])
	        {
	            return true;
	        }
	    }
	    $j = $i;
	}
	return $output;
}