Ошибка java недостаточно памяти

Если вы словили OutOfMemoryError, то это вовсе не значит, что ваше приложение создает много объектов, которые не могут почиститься сборщиком мусора и заполняют всю память, выделенную вами с помощью параметра -Xmx. Я, как минимум, могу придумать два других случая, когда вы можете увидеть эту ошибку. Дело в том, что память java процесса не ограничивается областью -Xmx, где ваше приложение программно создает объекты.

Область памяти, занимаемая java процессом, состоит из нескольких частей. Тип OutOfMemoryError зависит от того, в какой из них не хватило места.

1. java.lang.OutOfMemoryError: Java heap space

Не хватает место в куче, а именно, в области памяти в которую помещаются объекты, создаваемые программно в вашем приложении. Размер задается параметрами -Xms и -Xmx. Если вы пытаетесь создать объект, а места в куче не осталось, то получаете эту ошибку. Обычно проблема кроется в утечке памяти, коих бывает великое множество, и интернет просто пестрит статьями на эту тему.

2. java.lang.OutOfMemoryError: PermGen space

Данная ошибка возникает при нехватке места в Permanent области, размер которой задается параметрами -XX:PermSize и -XX:MaxPermSize. Что там лежит и как бороться с OutOfMemoryError возникающей там, я уже описал подробнейшим образом тут.

3. java.lang.OutOfMemoryError: GC overhead limit exceeded

Данная ошибка может возникнуть как при переполнении первой, так и второй областей. Связана она с тем, что памяти осталось мало и GC постоянно работает, пытаясь высвободить немного места. Данную ошибку можно отключить с помощью параметра -XX:-UseGCOverheadLimit, но, конечно же, её надо не отключать, а либо решать проблему утечки памяти, либо выделять больше объема, либо менять настройки GC.

4. java.lang.OutOfMemoryError: unable to create new native thread

Впервые я столкнулся с данной ошибкой несколько лет назад, когда занимался нагрузочным тестированием и пытался выяснить максимальное количество пользователей, которые могут работать с нашим веб-приложением. Я использовал специальную тулзу, которая позволяла логинить пользователей и эмулировать их стандартные действия. На определенном количестве клиентов, я начал получать OutOfMemoryError. Не особо вчитываясь в текст сообщения и думая, что мне не хватает памяти на создание сессии пользователя и других необходимых объектов, я увеличил размер кучи приложения (-Xmx). Каково же было мое удивление, когда после этого количество пользователей одновременно работающих с системой только уменьшилось. Давайте подробно разберемся как же такое получилось.

На самом деле это очень просто воспроизвести на windows на 32-битной машине, так как там процессу выделяется не больше 2Гб.

Допустим у вас есть приложение с большим количеством одновременно работающих пользователей, которое запускается с параметрами -Xmx1024M -XX:MaxPermSize=256M -Xss512K. Если всего процессу доступно 2G, то остается свободным еще коло 768M. Именно в данном остатке памяти и создаются стеки потоков. Таким образом, примерно вы можете создать не больше 768*(1024/512)=1536 (у меня при таких параметрах получилось создать 1316) нитей (см. рисунок в начале статьи), после чего вы получите OutOfMemoryError. Если вы увеличиваете -Xmx, то количество потоков, которые вы можете создать соответственно уменьшается. Вариант с уменьшением -Xss, для возможности создания большего количества потоков, не всегда выход, так как, возможно, у вас существуют в системе потоки требующие довольно больших стеков. Например, поток инициализации или какие-нибудь фоновые задачи. Но все же выход есть. Оказывается при программном создании потока, можно указать размер стека: Thread(ThreadGroup group, Runnable target, String name,long stackSize). Таким образом вы можете выставить -Xss довольно маленьким, а действия требующие больших стеков, выполнять в отдельных потоках, созданных с помощью упомянутого выше конструктора.

Более подробно, что же лежит в стеке потока, и куда уходит эта память, можно прочитать тут.

Конечно, вам может показаться данная проблема слегка надуманной, так как большинство серверов нынче крутиться на 64-битной архитектуре, но все же считаю данный пример весьма полезным, так как он помогает разобраться из каких частей состоит память java-процесса.

Powers of two grows exponentially. If vowelCount is high, one array alone could easily cause OutOfMemoryError (2^32 = 4GB).

You can try to tweak your VM maximum memory requirement (e.g. -Xmx512m), but do realize that your algorithm is requiring A LOT OF MEMORY. You may want to find a better algorithm if at all possible.

See also

• Wikipedia: exponential growth
  • Wheat and chessboard problem — famous anecdote to show just how surprisingly quick powers of two can grow
• java application launcher command line options
  • «-Xmxn : Specify the maximum size of the memory allocation pool.»

After edit: just as I expected, you’re generating a huge array filled with all binary possibilities. You rarely need to actually store this whole array in memory. You can just generate each possible combination «on-the-fly» and feed it to whoever needs the 0s and 1s «just-in-time».

Do keep in mind that this is still exponential growth, so even though you’ve taken care of your memory requirement from O(2^N) to just O(N), your time complexity is still O(2^N).

each time this method is called, that array is getting created. Is it possible that the array is not getting released .

Yes, that is very possible, if the reference to the array is ever leaked, and then something somewhere holds on to this reference. The garbage collector doesn’t really care what you think is/isn’t garbage; as long as an object is referred to by something (and it’s not a weak reference etc), it’s NOT garbage.

After figuring out what you’re trying to do, here’s my solution. Note that it doesn’t generate an array of bits at all.

static void generate(String prefix, String suffix) {
    int i = suffix.replaceAll("[aeiou].*", "").length();
    if (i == suffix.length()) {
        System.out.println(prefix + suffix);
    } else {
        generate(prefix + suffix.substring(0, i), suffix.substring(i + 1));
        generate(prefix + suffix.substring(0, i+1), suffix.substring(i + 1));
    }
}

// generate("", "apple");

It uses regex to find where the next vowel is. You can use a regular for-loop instead and the general algorithm will still work. You can optimize it to use StringBuilder instead (I’m mostly going for conciseness and hopefully clarity in this snippet).

Here’s an alternative solution that uses split to pre-chop the input string into pieces (O(N) space), then uses a StringBuilder to generate all the other strings (O(N) space).

static void generate(StringBuilder sb, String[] parts, int i) {
    if (i == parts.length) {
        System.out.println(sb.toString());
    } else {
        if ("aeiou".contains(parts[i])) {
            generate(sb, parts, i + 1);
        }
        sb.append(parts[i]);
        generate(sb, parts, i + 1);
        sb.setLength(sb.length() - parts[i].length());
    }
}
static void generate(String s) {
    generate(
        new StringBuilder(),
        s.split("(?<=[aeiou])|(?=(?!^)[aeiou])"),
        0
    );
}

// generate("apple");

The regex splits "apple" into [ "a", "ppl", "e" ]. It splits everywhere after a vowel, or (if it’s not the beginning of the string) everywhere before a vowel.

It should be obvious now that the space requirement is O(N), so unless your string is ridiculously long, this should not cause OutOfMemoryError.

Of course, if you’re storing the generated strings — all O(2^N) of them — in memory then of course you’ll get OutOfMemoryError. I hope this fact is obvious.

The entire idea is to not store in memory anything that you don’t need to generate this HUGE OUTPUT. If you then store all of this HUGE OUTPUT in memory (instead of, say, printing them to stdout or a file) then it defeats the whole purpose and you’ll get an OutOfMemoryError as expected.

Powers of two grows exponentially. If vowelCount is high, one array alone could easily cause OutOfMemoryError (2^32 = 4GB).

You can try to tweak your VM maximum memory requirement (e.g. -Xmx512m), but do realize that your algorithm is requiring A LOT OF MEMORY. You may want to find a better algorithm if at all possible.

See also

• Wikipedia: exponential growth
  • Wheat and chessboard problem — famous anecdote to show just how surprisingly quick powers of two can grow
• java application launcher command line options
  • «-Xmxn : Specify the maximum size of the memory allocation pool.»

After edit: just as I expected, you’re generating a huge array filled with all binary possibilities. You rarely need to actually store this whole array in memory. You can just generate each possible combination «on-the-fly» and feed it to whoever needs the 0s and 1s «just-in-time».

Do keep in mind that this is still exponential growth, so even though you’ve taken care of your memory requirement from O(2^N) to just O(N), your time complexity is still O(2^N).

each time this method is called, that array is getting created. Is it possible that the array is not getting released .

Yes, that is very possible, if the reference to the array is ever leaked, and then something somewhere holds on to this reference. The garbage collector doesn’t really care what you think is/isn’t garbage; as long as an object is referred to by something (and it’s not a weak reference etc), it’s NOT garbage.

After figuring out what you’re trying to do, here’s my solution. Note that it doesn’t generate an array of bits at all.

static void generate(String prefix, String suffix) {
    int i = suffix.replaceAll("[aeiou].*", "").length();
    if (i == suffix.length()) {
        System.out.println(prefix + suffix);
    } else {
        generate(prefix + suffix.substring(0, i), suffix.substring(i + 1));
        generate(prefix + suffix.substring(0, i+1), suffix.substring(i + 1));
    }
}

// generate("", "apple");

It uses regex to find where the next vowel is. You can use a regular for-loop instead and the general algorithm will still work. You can optimize it to use StringBuilder instead (I’m mostly going for conciseness and hopefully clarity in this snippet).

Here’s an alternative solution that uses split to pre-chop the input string into pieces (O(N) space), then uses a StringBuilder to generate all the other strings (O(N) space).

static void generate(StringBuilder sb, String[] parts, int i) {
    if (i == parts.length) {
        System.out.println(sb.toString());
    } else {
        if ("aeiou".contains(parts[i])) {
            generate(sb, parts, i + 1);
        }
        sb.append(parts[i]);
        generate(sb, parts, i + 1);
        sb.setLength(sb.length() - parts[i].length());
    }
}
static void generate(String s) {
    generate(
        new StringBuilder(),
        s.split("(?<=[aeiou])|(?=(?!^)[aeiou])"),
        0
    );
}

// generate("apple");

The regex splits "apple" into [ "a", "ppl", "e" ]. It splits everywhere after a vowel, or (if it’s not the beginning of the string) everywhere before a vowel.

It should be obvious now that the space requirement is O(N), so unless your string is ridiculously long, this should not cause OutOfMemoryError.

Of course, if you’re storing the generated strings — all O(2^N) of them — in memory then of course you’ll get OutOfMemoryError. I hope this fact is obvious.

The entire idea is to not store in memory anything that you don’t need to generate this HUGE OUTPUT. If you then store all of this HUGE OUTPUT in memory (instead of, say, printing them to stdout or a file) then it defeats the whole purpose and you’ll get an OutOfMemoryError as expected.

Overview

An out of memory error in Java formally known as java.lang.OutOfMemoryError is a runtime error that occurs when the Java Virtual Machine (JVM) cannot allocate an object in the Java heap memory. In this article, we will be discussing several reasons behind “out of memory” errors in Java and how you can avoid them.

The JVM manages the memory by setting aside a specific size of the heap memory to store the newly allocated objects. All the referenced objects remain active in the heap and keep that memory occupied until their reference is closed. When an object is no longer referenced, it becomes eligible to be removed from the heap by the Garbage collector to free up the occupied heap memory. In certain cases, the Java Garbage Collector (GC) is unable to free up the space required for a new object and the available heap memory is insufficient to support the loading of a Java class, this is when an “out of memory” error occurs in Java.

What causes the out of memory error in Java?

An “out of memory” error in Java is not that common and is a direct indication that something is wrong in the application. For instance, the application code could be referencing large objects for too long that is not required or trying to process large amounts of data at a time. It is even possible that the error could have nothing to do with objects on the heap and the reason behind it like because of third-party libraries used within an application or due to an application server that does not clean up after deployment.

Following are some of the main causes behind the unavailability of heap memory that cause the out of memory error in Java.

· Java heap space error

It is the most common out of memory error in Java where the heap memory fills up while unable to remove any objects.

See the code snippet below where java.lang.OutOfMemoryError is thrown due to insufficient Java heap memory available:

public class OutOfMemoryError01 {
    public static void main(String[] args) {
        Integer[] arr = new Integer[1000 * 1000 * 1000];
    }
}

Output:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at OutOfMemoryErrorExample.main(OutOfMemoryErrorExample.java:8)

In the above code, an array of integers with a very large size is attempted to be initialized. As the Java heap is insufficient to allocate such a huge array, it will eventually throw a java.lang.OutOfMemoryError: Java heap space error. Initially, it might seem fine but over time, it will result in consuming a lot of Java heap space and when it fills all of the available memory in the heap, Garbage Collection will not be able to clean it as the code would still be in execution and the no memory can be freed.

Another reason for a Java heap space error is the excessive use of finalizers. If a class has a finalize() method, the GC will not clean up any objects of that class, instead, they all will be queued up for finalization at a later stage. If a finalizer thread cannot keep up with the finalization queue because of excessive usage of finalizers, the Java heap will eventually fill up resulting in an “out of memory” error in Java.

Prevention:

Developers need to use the finalize methods only when required and they must monitor all the objects for which finalization would be pending.

· GC Overhead limit exceeded:

This error indicates that the garbage collector is constantly running due to which the program will also be running very slowly. In a scenario where for minimum consecutive 5 garbage collection cycles, if a Java process utilizes almost 98% of its time for garbage collection and could recover less than 2% of the heap memory then a Java Out of Memory Error will be thrown.
This error typically occurs because the newly generated data could barely fit into the Java heap memory having very little free space for new object allocations.

Prevention:

Java developers have the option to set the heap size by themselves. To prevent this error, you must Increase the heap size using the -Xmx attribute when launching the JVM.

· PermGen space error:

JVM separates the memory into different sections. One of the sections is Permanent Generation (PermGen) space. It is used to load the definitions of new classes that are generated at the runtime. The size of all these sections, including the PermGen area, is set at the time of the JVM launch. If you do not set the sizes of every area yourself, platform-specific defaults sizes will be then set. If the Permanent Generation’s area is ever exhausted, it will throw the java.lang.OutOfMemoryError: PermGen space error.

Prevention:

The solution to this out of Memory Error in Java is fairly simple. The application just needs more memory to load all the classes to the PermGen area so just like the solution for GC overhead limit exceeding error, you have to increase the size of the PermGen region at the time of Java launch. To do so, you have to change the application launch configuration and increase or if not used, add the –XX:MaxPermSize parameter to your code.

· Out of MetaSpace error:

All the Java class metadata is allocated in native memory (MetaSpace). The amount of MetaSpace memory to be used for class metadata is set by the parameter MaxMetaSpaceSize. When this amount exceeds, a java.lang.OutOfMemoryError exception with a detail MetaSpace is thrown.

Prevention:

If you have set the MaxMetaSpaceSize on the command line, increasing its size manually can solve the problem. Alternatively, MetaSpace is allocated from the same address spaces as the Java heap memory so by reducing the size of the Java heap, you can automatically make space available for MetaSpace. It should only be done when you have excess free space in the Java heap memory or else you can end up with some other Java out of memory error.

· Out of swap space error:

This error is often occurred due to certain operating system issues, like when the operating system has insufficient swap space or a different process running on the system is consuming a lot of memory resources.

Prevention:

There is no way to prevent this error as it has nothing to do with heap memory or objects allocation. When this error is thrown, the JVM invokes the error handling mechanism for fatal errors. it generates an error log file, which contains all the useful information related to the running threads, processes, and the system at the time of the crash. this log information can be very useful to minimize any loss of data.

How to Catch java.lang.OutOfMemoryError?

As the java.lang.OutOfMemoryError is part of the Throwable class, it can be caught and handled in the application code which is highly recommended. The handling process should include the clean up the resources, logging the last data to later identify the reason behind the failure, and lastly, exit the program properly.

See this code example below:

public class OutOfMemoryError02 {
    public void createArr (int size) {
        try {
            Integer[] myArr = new Integer[size];
        } catch (OutOfMemoryError ex) {
            //creating the Log
            System.err.println("Array size is too large");
            System.err.println("Maximum JVM memory: " + 
Runtime.getRuntime().maxMemory());
        }
    }
    public static void main(String[] args) {
        OutOfMemoryError02 oomee = new OutOfMemoryError02();
        ex.createArr (1000 * 1000 * 1000);
    }
}

In the above code, as the line of code that might cause an out of Memory Error is known, it is handled using a try-catch block. In case, if the error occurs, the reason for the error will be logged that is the large size of the array and the maximum size of the JVM, which will be later helpful for the caller of the method to take the action accordingly.

In case of an out of memory error, this code will exit with the following message:

Array size is too large
Maximum JVM memory: 9835679212

It is also a good option to handle an out of Memory Error in Java when the application needs to stay in a constant state in case of the error. This allows the application to keep running normally if any new objects are not required to be allocated.

See Also: CompletableFuture In Java With Examples

Conclusion

In this article, we have extensively covered everything related to the “out of memory” error in Java. In most cases, you can now easily prevent the error or at least will be able to retrieve the required information after the crashing of the program to identify the reason behind it. Managing errors and exceptions in your code is always challenging but being able to understand and avoid these errors can help you in making your applications stable and robust.