mojoliciousのデプロイ

会社でmojoliciousを使って開発したサービスがあるのですが、
リリースが近づいてきたのでデプロイをどうするか決めることに。


perlのWAF使ったデプロイをしたことがないので
どの方法がいいか決めかねている。
そのためいろいろ試してみて問題なさそうなのを採用。

デプロイ参考

MojoのCookBookを参考にしてみる*1
http://github.com/kraih/mojo/blob/master/lib/Mojolicious/Guides/Cookbook.pod
ただし、nginxとfast-cgiは除外。
nginxは使ったことない。fcgiは設定うまく行かなかった


サーバは貧弱

なのでなるだけリソースを使わない方法がいいかも*2

調査方法

apachebenchで負荷かけてその様子をみる。
本当はvmstatとかsarとか見るべきかもしれないが、
今回はtopだけでなんとなく眺めるだけにする。
意味ないかもしれないけど、topはスナップショットを張り付けてみる

サーバ環境(vmware server)

os CentOS 5.3 64bit
メモリ 1G
CPUコア 2
perl 5.12.1
apache 2.2.15
mojolicious 0.999926

テストアプリ

mojolicious generate app Demo2

で生成したアプリで「/」にアクセス。

テスト1:mojodaemon

コマンド

perl script/demo2 daemon

ab

ab -n 3000 -c 10 http://192.168.126.138:3000/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.126.138 (be patient)
Completed 300 requests
...
Finished 3000 requests

Server Software:        Mojolicious
Server Hostname:        192.168.126.138
Server Port:            3000

Document Path:          /
Document Length:        383 bytes

Concurrency Level:      10
Time taken for tests:   17.74414 seconds
Complete requests:      3000
Failed requests:        0
Write errors:           0
Total transferred:      1698000 bytes
HTML transferred:       1149000 bytes
Requests per second:    175.70 [#/sec] (mean)
Time per request:       56.915 [ms] (mean)
Time per request:       5.691 [ms] (mean, across all concurrent requests)
Transfer rate:          97.10 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.8      0       7
Processing:    23   55   5.8     55      85
Waiting:       21   54   5.7     54      84
Total:         30   56   5.8     56      86

Percentage of the requests served within a certain time (ms)
  50%     56
  66%     58
  75%     59
  80%     60
  90%     63
  95%     66
  98%     71
  99%     76
 100%     86 (longest request)

top

top - 09:41:58 up  6:19,  2 users,  load average: 3.46, 2.88, 1.38
Tasks: 124 total,   2 running, 122 sleeping,   0 stopped,   0 zombie
Cpu(s): 42.6%us,  1.5%sy,  0.0%ni, 49.5%id,  0.5%wa,  3.6%hi,  2.3%si,  0.0%st
Mem:   1026824k total,   809916k used,   216908k free,   147508k buffers
Swap:  2064376k total,        0k used,  2064376k free,   455588k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 9070 root      19   0  108m  14m 2020 R 100.7  1.4   0:13.80 perl
 9063 root      15   0 12740 1092  800 R  0.3  0.1   0:00.41 top
    1 root      15   0 10348  696  584 S  0.0  0.1   0:01.97 init
    2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.06 migration/0
    3 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/0
    4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
...

スト2mojodaemon_prefork

コマンド

perl script/demo2 daemon_prefork --clients 1000 --servers 10

ab

ab -n 3000 -c 10 http://192.168.126.138:3000/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.126.138 (be patient)
Completed 300 requests
...
Finished 3000 requests


Server Software:        Mojolicious
Server Hostname:        192.168.126.138
Server Port:            3000

Document Path:          /
Document Length:        383 bytes

Concurrency Level:      10
Time taken for tests:   43.170359 seconds
Complete requests:      3000
Failed requests:        0
Write errors:           0
Total transferred:      1698000 bytes
HTML transferred:       1149000 bytes
Requests per second:    69.49 [#/sec] (mean)
Time per request:       143.901 [ms] (mean)
Time per request:       14.390 [ms] (mean, across all concurrent requests)
Transfer rate:          38.41 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    8 144.9      1    3005
Processing:     5  135 275.0     46    2659
Waiting:        4  132 272.5     44    2657
Total:          5  143 309.4     46    3088

Percentage of the requests served within a certain time (ms)
  50%     46
  66%     54
  75%    121
  80%    179
  90%    258
  95%    604
  98%   1304
  99%   1638
 100%   3088 (longest request)

top

top - 09:45:37 up  6:22,  2 users,  load average: 7.07, 3.44, 1.79
Tasks: 134 total,  11 running, 123 sleeping,   0 stopped,   0 zombie
Cpu(s): 96.3%us,  3.4%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.3%si,  0.0%st
Mem:   1026824k total,   872764k used,   154060k free,   147804k buffers
Swap:  2064376k total,        0k used,  2064376k free,   456820k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 9079 root      19   0  111m  14m 1460 R 37.6  1.5   0:16.53 perl
 9081 root      19   0  111m  14m 1448 R 34.3  1.5   0:18.13 perl
 9078 root      19   0  111m  14m 1460 R 32.9  1.5   0:13.59 perl
 9085 root      18   0  111m  14m 1448 R 23.3  1.5   0:12.00 perl
 9083 root      18   0  111m  14m 1448 R 17.6  1.5   0:09.57 perl
 9076 root      18   0  111m  14m 1460 R 14.0  1.5   0:17.44 perl
 9077 root      19   0  111m  14m 1460 R 11.0  1.5   0:12.23 perl
 9080 root      19   0  111m  14m 1460 R 10.0  1.5   0:20.60 perl
 9082 root      20   0  111m  14m 1448 R  9.6  1.5   0:09.23 perl
 9084 root      19   0  111m  14m 1448 R  6.7  1.5   0:09.63 perl
 9075 root      15   0  110m  14m 2040 S  0.7  1.5   0:01.38 perl
    1 root      15   0 10348  696  584 S  0.0  0.1   0:01.97 init
    2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.06 migration/0
...

テスト3:mod_perl

httpd.conf

<IfModule prefork.c>
StartServers      10
MinSpareServers   10
MaxSpareServers   10
ServerLimit       10
MaxClients        10
MaxRequestsPerChild  4000
</IfModule>

KeepAlive Off

PerlRequire /usr/local/apache2/startup.pl
LoadModule perl_module modules/mod_perl.so
# plack
<VirtualHost *:80>
    ServerName centos.www2
    DocumentRoot /usr/local/sandbox

    <Perl>
        use Plack::Handler::Apache2;
        Plack::Handler::Apache2->preload("/usr/local/sandbox/demo2/script/demo2.psgi");

        $ENV{PERL5LIB} = '/usr/local/sandbox/demo2/lib';
        $ENV{PLACK_ENV} = 'production';
        $ENV{MOJO_HOME} = '/usr/local/sandbox/demo2';
    </Perl>

    <Location /mojo-modperl>
        SetHandler perl-script
        PerlHandler Plack::Handler::Apache2
        PerlSetVar psgi_app /usr/local/sandbox/demo2/script/demo2.psgi
    </Location>
</VirtualHost>

ab

ab -n 3000 -c 10 http://192.168.126.138/mojo-modperl
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.126.138 (be patient)
Completed 300 requests
...
Finished 3000 requests


Server Software:        Apache/2.2.15
Server Hostname:        192.168.126.138
Server Port:            80

Document Path:          /mojo-modperl
Document Length:        396 bytes

Concurrency Level:      10
Time taken for tests:   12.492525 seconds
Complete requests:      3000
Failed requests:        0
Write errors:           0
Total transferred:      1881000 bytes
HTML transferred:       1188000 bytes
Requests per second:    240.14 [#/sec] (mean)
Time per request:       41.642 [ms] (mean)
Time per request:       4.164 [ms] (mean, across all concurrent requests)
Transfer rate:          146.97 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   1.2      1       6
Processing:     7   39 167.8     29    7199
Waiting:        6   37 167.7     27    7198
Total:          9   40 167.8     30    7199

Percentage of the requests served within a certain time (ms)
  50%     30
  66%     33
  75%     35
  80%     36
  90%     43
  95%     76
  98%    130
  99%    177
 100%   7199 (longest request)

top

top - 10:09:22 up  6:46,  2 users,  load average: 3.04, 2.56, 2.43
Tasks: 128 total,  13 running, 115 sleeping,   0 stopped,   0 zombie
Cpu(s): 31.0%us,  2.1%sy,  0.0%ni, 62.9%id,  0.0%wa,  1.2%hi,  2.8%si,  0.0%st
Mem:   1026824k total,   872804k used,   154020k free,   149516k buffers
Swap:  2064376k total,        0k used,  2064376k free,   465756k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 9349 daemon    16   0  148m  15m 2532 R 10.3  1.6   0:03.77 httpd
 9350 daemon    15   0  148m  15m 2532 R  9.6  1.6   0:03.90 httpd
 9353 daemon    16   0  148m  15m 2532 R  9.3  1.6   0:03.39 httpd
 9347 daemon    15   0  148m  15m 2532 R  9.0  1.6   0:03.14 httpd
 9348 daemon    15   0  148m  15m 2532 R  8.6  1.6   0:03.36 httpd
 9345 daemon    16   0  148m  15m 2532 R  8.0  1.6   0:02.17 httpd
 9352 daemon    16   0  148m  15m 2532 R  7.0  1.6   0:03.46 httpd
 9354 daemon    16   0  148m  15m 2532 R  6.6  1.6   0:03.12 httpd
 9346 daemon    16   0  148m  15m 2532 R  4.6  1.6   0:02.03 httpd
 9356 root      15   0 12740 1088  800 R  0.3  0.1   0:00.27 top
    1 root      15   0 10348  696  584 S  0.0  0.1   0:01.97 init
    2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.07 migration/0
...

テスト4:plackup HTTP::Server::PSGI

psgiファイルを手動生成*3
vi script/demo2.psgi

# myapp.psgi
use Mojo::Server::PSGI;
my $psgi = Mojo::Server::PSGI->new(app_class => 'demo2');
my $app  = sub { $psgi->run(@_) };


コマンド

plackup -E production script/demo2.psgi

ab

 ab -n 3000 -c 10 http://192.168.126.138:5000/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.126.138 (be patient)
Completed 300 requests
...
Finished 3000 requests


Server Software:        HTTP::Server::PSGI
Server Hostname:        192.168.126.138
Server Port:            5000

Document Path:          /
Document Length:        383 bytes

Concurrency Level:      10
Time taken for tests:   15.877629 seconds
Complete requests:      3000
Failed requests:        0
Write errors:           0
Total transferred:      1752000 bytes
HTML transferred:       1149000 bytes
Requests per second:    188.95 [#/sec] (mean)
Time per request:       52.925 [ms] (mean)
Time per request:       5.293 [ms] (mean, across all concurrent requests)
Transfer rate:          107.70 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.7      0       7
Processing:    27   51   9.0     52     167
Waiting:       26   50   8.9     51     165
Total:         27   52   9.2     52     169

Percentage of the requests served within a certain time (ms)
  50%     52
  66%     55
  75%     56
  80%     57
  90%     60
  95%     63
  98%     66
  99%     70
 100%    169 (longest request)

top

top - 10:11:20 up  6:48,  2 users,  load average: 1.14, 2.21, 2.33
Tasks: 129 total,   2 running, 127 sleeping,   0 stopped,   0 zombie
Cpu(s): 45.6%us,  2.1%sy,  0.0%ni, 42.8%id,  0.0%wa,  5.2%hi,  4.2%si,  0.0%st
Mem:   1026824k total,   828232k used,   198592k free,   149524k buffers
Swap:  2064376k total,        0k used,  2064376k free,   467732k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 9380 root      25   0  115m  15m 2076 R 101.5  1.6   0:07.93 plackup
 9381 root      15   0 12740 1076  800 R  1.0  0.1   0:00.15 top
 1866 root      18   0 10760  372  244 S  0.3  0.0   0:01.92 irqbalance
    1 root      15   0 10348  696  584 S  0.0  0.1   0:01.97 init
    2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.07 migration/0
...

テスト5:plackup Starlet

コマンド

plackup -E production -s Starlet --max-workers=10 --max-keepalive-reqs=1 --max-reqs-per-child=10000 script/demo2.psgi

ab

ab -n 3000 -c 10 http://192.168.126.138:5000/
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.126.138 (be patient)
Completed 300 requests
...
Finished 3000 requests


Server Software:        Plack::Handler::Starlet
Server Hostname:        192.168.126.138
Server Port:            5000

Document Path:          /
Document Length:        383 bytes

Concurrency Level:      10
Time taken for tests:   14.753736 seconds
Complete requests:      3000
Failed requests:        0
Write errors:           0
Total transferred:      1767000 bytes
HTML transferred:       1149000 bytes
Requests per second:    203.34 [#/sec] (mean)
Time per request:       49.179 [ms] (mean)
Time per request:       4.918 [ms] (mean, across all concurrent requests)
Transfer rate:          116.92 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   1.2      1       8
Processing:     3   46 545.9     14   14746
Waiting:        2   43 545.4     12   14742
Total:          4   48 546.0     16   14751

Percentage of the requests served within a certain time (ms)
  50%     16
  66%     18
  75%     21
  80%     23
  90%     40
  95%     64
  98%    128
  99%    246
 100%  14751 (longest request)

top

top - 10:12:32 up  6:49,  2 users,  load average: 1.61, 2.02, 2.25
Tasks: 139 total,  16 running, 123 sleeping,   0 stopped,   0 zombie
Cpu(s): 83.6%us,  6.4%sy,  0.0%ni,  0.0%id,  0.0%wa,  2.7%hi,  7.4%si,  0.0%st
Mem:   1026824k total,   921452k used,   105372k free,   149548k buffers
Swap:  2064376k total,        0k used,  2064376k free,   468808k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 9390 root      17   0  118m  15m 1704 R 57.0  1.6   0:03.11 plackup
 9386 root      16   0  118m  15m 1704 R 46.7  1.6   0:05.50 plackup
 9394 root      16   0  118m  15m 1704 R 41.7  1.6   0:02.67 plackup
 9392 root      16   0  118m  15m 1704 R 41.1  1.6   0:01.86 plackup
 9388 root      18   0  111m  12m 1468 R  3.3  1.3   0:00.30 plackup
 9391 root      18   0  109m  10m 1444 R  3.3  1.1   0:00.33 plackup
 9393 root      18   0  109m  10m 1444 R  3.3  1.1   0:00.33 plackup
 9389 root      18   0  107m 9304 1424 R  1.3  0.9   0:00.25 plackup
  383 root      10  -5     0    0    0 S  0.3  0.0   0:02.28 kjournald
 9381 root      15   0 12740 1084  800 R  0.3  0.1   0:00.30 top
    1 root      15   0 10348  696  584 S  0.0  0.1   0:01.97 init
...

テスト6:plackup Starman

何故か500エラーが出るので断念…たぶんケアレスミスだと思うけど



考察

abは何回かやってみた結果はStarletが一番早いような。
mod_perlもあまり変わらないけど。
しかしテスト2daemon_prefork)が異常に遅いのはなんでだろう?設定?
なのでStarletかmod_perlにしようとおもうけど、
運用のしやすさを考えるとmod_perlになるかなー?*4

*1:cpanにはまだ上がってないみたい

*2:識者の方々ならすぐに思いつくんだろうな

*3:http://perl-users.jp/articles/advent-calendar/2009/casual/23.html ここで書かれているようなヘルパは使えなかった…

*4:単にStarletの運用をしらないだけ