最近收到生产环境的Tomcat突然挂掉的报警，于是感觉排查，一番操作之后发现是内存溢出了，解决方案就是服务器升级内存或者迁移走那些占用内存比较大的应用。

排查步骤：

• 先查看系统报错日志（/var/log/messages）：

grep -i 'killed process' /var/log/messages #发现java进程被kill掉了

Feb  1 00:10:56 web10 kernel: Killed process 8934 (java) total-vm:6714352kB, anon-rss:2931564kB, file-rss:0kB, shmem-rss:0kB

• 进一步查看上下文错误：

grep -C 5 'Feb  1 00:10:56 web10 kernel: Killed process 8934' messages #查看错误行的前后5行错误

Feb  1 00:10:56 web2 kernel: [25510]     0 25510     4450      117      13        0             0 assist_daemon
Feb  1 00:10:56 web2 kernel: [ 2029]     0  2029  1274523   163798     580        0             0 java
Feb  1 00:10:56 web2 kernel: [30991]     0 30991  1011441   147992     419        0             0 java
Feb  1 00:10:56 web2 kernel: [31804]     0 31804   980370    94910     283        0             0 java
Feb  1 00:10:56 web2 kernel: Out of memory: Kill process 8934 (java) score 175 or sacrifice child #最后发现是内存溢出被kill掉了
Feb  1 00:10:56 web2 kernel: Killed process 8934 (java) total-vm:6714352kB, anon-rss:2931564kB, file-rss:0kB, shmem-rss:0kB
Feb  1 00:10:56 web2 systemd-logind: Removed session 148705.
Feb  1 00:20:02 web2 systemd: Started Session 149280 of user root.
Feb  1 00:30:01 web2 systemd: Started Session 149282 of user root.
Feb  1 00:30:01 web2 systemd: Started Session 149281 of user root.
Feb  1 00:40:01 web2 systemd: Started Session 149283 of user root.

转载请注明：扩产网 » tomcat突然宕机问题排查