At-home experiment*!

Question

23

Memory limiting solutions for greedy applications that can crash OS?

rated 0 times [ 23] [ 0] / answers: 1 / hits: 30054 / 2 Years ago, sat, march 19, 2022, 5:00:53

I use my computer for scientific programming. It has a healthy 8GB of RAM and 12GB of swap space. Often, as my problems have gotten larger, I exceed all of the available RAM. Rather than crashing (which would be preferred), it seems Ubuntu starts loading everything into swap, including Unity and any open terminals. If I don't catch a run-away program in time, there is nothing I can do but wait - it takes 4-5 minutes to switch to a command prompt eg. Ctrl-Alt-F2 where I can kill the offending process.

Since my own stupidity is out of scope of this forum, how can I prevent Ubuntu from crashing via thrashing when I use up all of the available memory from a single offending program?

At-home experiment*!

Open a terminal, launch python and if you have numpy installed try this:

>>> import numpy

>>> [numpy.zeros((10**4, 10**4)) for _ in xrange(50)]

* Warning: may have adverse effects, monitor the process via iotop or top to kill it in time. If not, I'll see you after your reboot.

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

gigglcept

Add To Favorites

Follow

Total Points: 113

Total Questions: 105

Total Answers: 125

Location: Christmas Island

Member since Wed, Jan 13, 2021

3 Years ago

answered 2 Years ago avilyexplor · Accepted Answer

The shell built-in ulimit allows you to restrict resources. For your case, to limit memory use in the shell (and its children), use ulimit -v.

Demonstration setting a memory limit of 100 MB (100000 KB):

$ ulimit -v

unlimited

$ python -c '[ "x" * 100000000 ]'

$ ulimit -v 100000

$ python -c '[ "x" * 100000000 ]'

Traceback (most recent call last):

  File "<string>", line 1, in <module>

MemoryError

It's observed using ps uww -C script-name-here that python requires at least 29MB of memory (VSZ column). The RSS limit grows as your python script needs more memory so adapt that column.