Problems in performance testing with Python

1

A group of friends and I are starting an initiative to create a repo in github in order to test the benchmark performance of http services implemented in several languages and frameworks. One of the languages we've implemented is Python, without using any third-party framework.

The implementation was basically like this:

#!/usr/bin/python
from BaseHTTPServer import BaseHTTPRequestHandler,HTTPServer

PORT_NUMBER = 3000

class serverHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.send_header('Content-type','text/html')
        self.end_headers()
        self.wfile.write('yo')
        return

try:
    #Create a web server and define the handler to manage the
    #incoming request
    server = HTTPServer(('', PORT_NUMBER), serverHandler)
    print 'Started httpserver on port ' , PORT_NUMBER

    #Wait forever for incoming htto requests
    server.serve_forever()

except KeyboardInterrupt:
    print '^C received, shutting down the web server'
    server.socket.close()

To perform the performance tests we use the boom ( link ) through the command:

boom -n 100000 -c 10000 http://localhost:3000

When we set off the boom against the service implemented in Python we have a disappointing performance, which should not happen at first.

Result:

Summary:
  Total:    25.5676 secs
  Slowest:  4.2101 secs
  Fastest:  0.9872 secs
  Average:  1.7560 secs
  Requests/sec: 380.0513

Status code distribution:
  [200] 9717 responses

Response time histogram:
  0.987 [1] |
  1.309 [1792]  |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  1.632 [2670]  |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  1.954 [1681]  |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  2.276 [2616]  |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  2.599 [446]   |∎∎∎∎∎∎
  2.921 [283]   |∎∎∎∎
  3.243 [171]   |∎∎
  3.565 [31]    |
  3.888 [1] |
  4.210 [25]    |

Latency distribution:
  10% in 1.0313 secs
  25% in 1.5239 secs
  50% in 1.7353 secs
  75% in 2.0630 secs
  90% in 2.2717 secs
  95% in 2.6155 secs
  99% in 3.1570 secs

I'd like to know what I've done wrong. The intention in this case is to be as simplistic as possible, without using allegories such as caching, or connection pooling control (unless this is implicit in the language).

For those who want to check other results and implementations and / or want to contribute to new implementations in languages and frameworks, feel free to. The codes and results of all implementations can be found at: link

    
asked by anonymous 06.04.2016 / 16:57

1 answer

0

In the sample code you can only process one request at a time (there is no parallelism). In other words, the Python interpreter will idle while waiting for the data to be sent to the client.

In computing terms, this waiting time is precious and could be used to process other requests - so when you analyze the performance of frameworks (such as Tornado , which uses a event loop ) you notice a large difference in the number of requests met.

Another factor is that since boom runs on the machine itself on which the server is running, competition for CPU resources occurs between the deployed server and the benchmark software (in this case, boom ) - which distorts the results. Even if the server ran on a different machine, you would have to consider other issues such as bandwidth, machine CPU that is firing the test, and operating system settings.

Any benchmark reflects only part of the reality - keep that in mind when comparing your results.

    
30.06.2016 / 04:20