月度归档:2019年09月

flask wtform中枚举的使用

wtform可以非常友好的管理好表单的交互以及服务端验证。最近开始使用postgres,发现也能非常友好的支持enum类型,但是在实际的操作中发现了一个bug。这里我们使用form.populate_obj方法从form中更新字段到obj,使用form.process(obj)方法从obj更新到form。sqlalchemy使用枚举类型定义的字段,form中使用selectfiled或者其子类。

修改了好多回,反反复复跟踪后才明确这个问题,在第一次给form赋值时无错,第一次保存时也无错,bug出在form的存储数据上,就是对field.data的操作。原来在wtform中会判断field.data是否是filed.choices中的值,源代码如下:

file:core.py    
def pre_validate(self, form):
        for v, _ in self.choices:
            if self.data == v:
                break
        else:
            raise ValueError(self.gettext('Not a valid choice'))

self.data有时候是form中选中input的value值,有时候是枚举类型,要修复这个bug就要在所有枚举类型的基础类BaseEnum上改写eq和hash两个内置函数,使得双等“==”操作既能比较字符串,也能比较枚举类型,修改后修复。

gunicorn vs uwsgi 性能对比,以flask run为基准

之前一直在用gunicorn,这次折腾新项目突发奇想来对比下flask run \gunicorn\uwsgi的性能。不是非常严谨的测试,希望也能有一定的参考意义,结果也很有意思。

gunicorn很火,uwsgi以前用的人多,但是现在少了感觉不怎么听到,这东西是用c写的,带了光环的。flask run这里只是作为基准参考,压力测试工具为ab test,前置了nginx把静态文件剥离了,两者都使用unix domain socket通信隔离了http微乎其微的影响,两者都是用了2process+2thread,压力为100-1000。下面是三种方式的结果:

#基准flask run 100-1000
Server Software:        nginx
Server Hostname:        xxx
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256
Document Path:          /
Document Length:        6223 bytes
Concurrency Level:      100
Time taken for tests:   7.418 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      6489000 bytes
HTML transferred:       6223000 bytes
Requests per second:    134.81 [#/sec] (mean)
Time per request:       741.790 [ms] (mean)
Time per request:       7.418 [ms] (mean, across all concurrent requests)
Transfer rate:          854.27 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        5   47  69.2      6     296
Processing:   231  662 186.7    679    1736
Waiting:      231  662 186.7    679    1736
Total:        241  709 159.6    702    1742
Percentage of the requests served within a certain time (ms)
  50%    702
  66%    735
  75%    788
  80%    821
  90%    881
  95%    958
  98%   1111
  99%   1178
 100%   1742 (longest request)
# uwsgi result
Server Software:        nginx
Server Hostname:        erp.ljmold.cn
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256
Document Path:          /user/login
Document Length:        6223 bytes
Concurrency Level:      100
Time taken for tests:   9.741 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      6489000 bytes
HTML transferred:       6223000 bytes
Requests per second:    102.66 [#/sec] (mean)
Time per request:       974.096 [ms] (mean)
Time per request:       9.741 [ms] (mean, across all concurrent requests)
Transfer rate:          650.54 [Kbytes/sec] received
Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        5  130 246.7      9    1750
Processing:     4  533 732.3    437    7584
Waiting:        4  533 732.3    437    7584
Total:          9  664 765.4    473    7742
Percentage of the requests served within a certain time (ms)
  50%    473
  66%    656
  75%    751
  80%    858
  90%   1272
  95%   1719
  98%   2526
  99%   4081
 100%   7742 (longest request)

#gunicorn 100-1000
 Server Software:        nginx
Server Hostname:        erp.ljmold.cn
Server Port:            443
SSL/TLS Protocol:       TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256

Document Path:          /user/login
Document Length:        6223 bytes

Concurrency Level:      100
Time taken for tests:   9.678 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      6489000 bytes
HTML transferred:       6223000 bytes
Requests per second:    103.33 [#/sec] (mean)
Time per request:       967.779 [ms] (mean)
Time per request:       9.678 [ms] (mean, across all concurrent requests)
Transfer rate:          654.79 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        4   82 233.2      6    2275
Processing:     4  584 891.2    306    7272
Waiting:        4  584 891.4    303    7272
Total:         10  667 907.5    389    7278

Percentage of the requests served within a certain time (ms)
  50%    389
  66%    573
  75%    682
  80%    868
  90%   1487
  95%   2506
  98%   3374
  99%   4788
 100%   7278 (longest request)
  1. 总共耗时上差的微乎其微,这点差别不能说明什么;
  2. 两者都完成了所有请求,这里说一下,之所以选1000就是因为2000两种方式都会崩溃;
  3. 平均时间uwsgi胜出,这里差别不明显,但是也说明总体性能上uwsgi做的彻底;
  4. 连接时间上,两者明显在1000下都吃力了,但是uwsgi表现更好,除了c我觉得使用uwsgi自己的协议也有关系;
  5. 最后也是最重要的,在高并发下,明显uwsgi的表现更好一点,很有意思的是在80%以下的正常流量范围内反倒gunicorn更好,而且效果显著,100ms还是比较可观的,我想这才是大家都用gunicorn的主要原因吧。

最后别人说的总归是片面,还是自己动手对性能有更全面的了解。顺便说一下对flask自带的小服务器刮目相看