Java on Linux crashed with the message 'There Is Insufficient Memory For The Java Runtime Environment To Continue. Cannot create GC thread. Out of system resources.'

Applies to

Java SE JDK and JRE - Version 6 and later
Linux x86-64
Linux x86

Symptoms

Out of Memory Errors are seen when trying to run Java processes in an environment. The processes are running correctly but once some limit is reached, the errors are seen and a new JVM is not started.

Snippet from the fatal error log. Important strings are indicated by ** :

#
# **There is insufficient memory for the Java Runtime Environment to continue.**
# **Cannot create GC thread. Out of system resources.**
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  **Out of Memory Error (gcTaskThread.cpp:46), pid=<PID>, tid=<TID>**
#
# JRE version: 6.0_35-b52
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.10-b01 mixed mode linux-amd64 compressed oops)
---------------  T H R E A D  ---------------
Current thread (0x00007f62d0007000):  JavaThread "Unknown thread" [_thread_in_vm, id=<ID>, stack(0x00007f62d6f4b000,0x00007f62d704c000)]
Stack: [0x00007f62d6f4b000,0x00007f62d704c000],  sp=0x00007f62d704a7c0,  free space=1021k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0x860395]  VMError::report_and_die()+0x265
V  [libjvm.so+0x3e5278]  report_vm_out_of_memory(char const*, int, unsigned long, char const*)+0x68
V  [libjvm.so+0x467a8a]  GCTaskThread::GCTaskThread(GCTaskManager*, unsigned, unsigned)+0x13a
V  [libjvm.so+0x46658e]  GCTaskManager::initialize()+0x21e
V  [libjvm.so+0x466323]  GCTaskManager::GCTaskManager(unsigned)+0x13
V  [libjvm.so+0x7245cd]  ParallelScavengeHeap::initialize()+0x4dd
V  [libjvm.so+0x837e29]  Universe::initialize_heap()+0xa9
V  [libjvm.so+0x8379ca]  universe_init()+0x7a
V  [libjvm.so+0x4ad51b]  init_globals()+0x4b
V  [libjvm.so+0x81e5c4]  Threads::create_vm(JavaVMInitArgs*, bool*)+0x214
V  [libjvm.so+0x51b790]  JNI_CreateJavaVM+0x80

Changes

The maximum number of processes or threads that a user can run has been set to 1024.

$ ulimit -u
max user processes              (-u) 1024

There are 1007 processes running on the server already. In this case, any process that will try to create more than 17 threads will fail (1024-1007=17).

$ ps -fLu [list of users] | tail -n +2 | wc -l
   1007

Note: in order to find out how many threads an app usually is running on Linux, you could enter: $ top -p <PID>

Cause

The JVM wasn’t able to create a new native thread (a GC thread) due to a lack of system resources. The max user processes limit was reached.

From the getrlimit(2) man page:

RLIMIT_NPROC The maximum number of processes (or, more precisely on Linux, threads) that can be created for the real user ID of the calling process. Upon encountering this limit, fork(2) fails with the error EAGAIN.

Same for pthread_create(3):

EAGAIN Insufficient resources to create another thread, or a system-imposed limit on the number of threads was encountered. The latter case may occur in two ways: the RLIMIT_NPROC soft resource limit (set via setrlimit(2)), which limits the number of process for a real user ID, was reached; or the kernel’s system-wide limit on the number of threads, /proc/sys/kernel/threads-max, was reached.

Solution

To resolve this issue, set a higher maximum user processes limit. This is an OS specific procedure. Please consult your system administrator or appropriate OS documentation for details.

In general, the ulimit -u command can be used to set a temporary value like this:

$ ulimit -u 2048

This will double the limit seen in the previous example. However, as soon as the user closes their session, this value will be lost and the default will be restored the next time they login.

To make a persistent change to the limit for a user, edit the /etc/security/limits.conf file. Again, see the relevant OS documentation or man pages for details on the file and the settings needed. tails on the file and the settings needed.

Last reviewed on Sat Feb 01 2025 00:00:00 GMT+0000 (Coordinated Universal Time)