C and large arrays

This is an old page from the time when we used Sun-computers. The information is still valid though.

Some of the assignments require that you use large arrays. This may be a problem in C on a Sun-computer (and perhaps on other systems as well). Consider the following program: 

#include <stdio.h>

#define _N 2000000

main()
{
  int k;
  double large_array[_N];

  for(k = 0; k < _N; k++)
    large_array[k] = 1;

  printf("Last %f\n", large_array[_N - 1]);

  return 0;
}

When we try to run it we get:

% cc stack_problems_1.c
% a.out
Segmentation fault

The reason is that large_array is allocated on the stack, which has a limited size. We can find out the size by using the command limit . Thus:

% limit  (works provided you use tcsh)
cputime       unlimited
filesize      unlimited
datasize      2097148 kbytes
stacksize     8192 kbytes
coredumpsize  0 kbytes
vmemoryuse    unlimited
descriptors   1024

So, the stack is limited to 8192 kbyte (8 Mbyte), but we need 2000000 * 8 / 1024 kbyte, i.e. 15625 kbyte (the stack is used for some other purposes as well so it must be a bit larger). So, let us increase the stack size and try again:

% limit stacksize 15700 kbytes
% a.out
Last 1.000000

Another way is to store the array in a segment in a.out. If we make large_array static, i.e. we have the type declaration:

static double large_array[_N];

our program will work with the default stack size. The array is now stored in the bss-segment.

% cc stack_problems_2.c
% limit stacksize
stacksize 8192 kbytes
% a.out
Last 1.000000

% size a.out
text   data       bss        dec      hex   filename
2589    252  16000000   16002841   f42f19      a.out

One drawback with static variables is that they exist for the lifetime of the program (even if we do not use the array). So, yet another way (common) is the use dynamic memory allocation (i.e. we use malloc / free) placing the array on the heap: 

#include <stdio.h>
#include <stdlib.h>

#define _N 2000000

int main()
{
  int k;
  double *large_array;

  if ( (large_array = malloc(_N * sizeof(double))) == NULL) {
    printf("Could not malloc large_array.\n");
    exit(1);
  }

  for(k = 0; k < _N; k++)
    large_array[k] = 1;

  printf("Last %f\n", large_array[_N - 1]);

  free(large_array);

  return 0;
}

Back