How to pass an integer file to a vector in C?

The problem of your question is caused by this line, inside the main function:

int p[qtd]; 

The point is that C can not dynamically calculate the size of vectors - you have to put a fixed size to compile the program. In case you still try to use the qtd BEFORE value to calculate its value, then even if it were a dynamic language like Javascript, the value of "qtd" would not be defined in that statement. The fact is that the compiler simply creates a vector of unknown size in this statement - and by chance, when you exceed 42 numbers in the vector, the thing explodes. But it could be happening with 2 numbers, or 10,000. Another answer here suggests changing the statement from p to a point where qtd is set - but that does not work in C - it just gave the allocated vector the luck to be larger over there. The value in declaration statements of type int p[NUMERO] is used when the program is compiled, and is already fixed when it is executed.

The right way to deal with this is with dynamic memory allocation: you declare your variable p only as an integer vector:

int p[];

(or int *p; - will be the same) After you know the number of elements in the array, you allocate a region in memory by calling malloc . Tests whether the return from malloc is not NULL . (In general, answers here in S.O. skip this test, but it is vital in systems that go into production and should be part of the thinking of anyone who programs in C).

malloc is defined in "stdlib.h", which has already been included, so after scrolling through the file and knowing the value of qtd, you can add these lines to make room for your data:

...
int qtd;
int *p;
// parte do programa para saber o valor de qtd
...
p = malloc(sizeof(int) * qtd);
if (p == NULL) {
    printf("Falha ao alocar memória.\n");
    exit(1);
}
// Pode continuar a execução do seu codigo original aqui
...

This will eliminate your error - I have not checked other problems. Some tips however:

Modern PCs run billions of operations per second - and C uses this potential at full speed. This means that sorting vectors with 100 numbers is something that will be done in a few milliseconds, maybe less than 1. As long as the program works, you can increase the numbers used and the file size to fit even better - on the order of 100,000 numbers - there maybe you'll see some difference.

Another thing, avoid using the so-called "system": this runs every other program outside of your program, and is not part of the C language. At some point college teachers in Brazil started using system("pause"); so that students who did not know how to use the terminal could look at the output of their programs, but this practice is horrible. To wait for a key, you can use gets() itself (but the key must be the ENTER) - and to erase the screen, print a series of "\ n" s - (this is what the CLS does - only which is another whole program just to do this): for (int i=0; i< 80; i++) printf("\r\n"); .

Using system for these things, in addition to the machine resources (which will still look like snapshots of code of this kind, although they are in the order of 100,000 to 1 million times more memory and executed instructions) your specific program for windows - while without it, the same program can work on Mac, Linux, Unix servers on the internet, etc ...

    

Follow your changed code with some suggestions:

void quicksort(int *P, int tam);
void mergesort(int p1[],int i, int j);
void merge(int p2[],int i1,int j1,int i2,int j2);

void mergesort(int p1[],int i, int j)
{
    clock_t inicio = clock();
    int meio;     
    if(i<j)
    {
        meio=(i+j)/2;
        mergesort(p1,i,meio);       
        mergesort(p1,meio+1,j);    
        merge(p1,i,meio,meio+1,j);
    }
    clock_t fim = clock();
    double gasto = difftime(fim,inicio)/CLOCKS_PER_SEC;  
    printf("\ntempo gasto: %f segundos\n", gasto);
}

void merge(int p2[],int i1,int j1,int i2,int j2)
{
    int aux[j2];
    int i,j,k;
    i=i1; 
    j=i2; 
    k=0;   
    while(i<=j1 && j<=j2)
    {
        if(p2[i]<p2[j])
            aux[k++]=p2[i++];
        else
            aux[k++]=p2[j++];
    }  
    while(i<=j1)
        aux[k++]=p2[i++];        
    while(j<=j2) 
        aux[k++]=p2[j++];
    for(i=i1,j=0;i<=j2;i++,j++)
        p2[i]=aux[j];
}

void quicksort(int *P, int tam) {
    clock_t inicio = clock(); 
    if (tam < 2) {
        return;
    }
    int pivo = P[tam/2];
    int i3, j3;
    for (i3 = 0, j3 = tam - 1; ; i3++, j3--) {
        while (P[i3] < pivo){
            i3++;
        }
        while (P[j3] > pivo){
            j3--;
        }
        if (i3 >= j3) {
            break;
        }
        int aux2 = P[i3];
        P[i3] = P[j3];
        P[j3] = aux2;
    }
    quicksort(P, i3);
    quicksort(P + i3, tam - i3);
    clock_t fim = clock();
    double gasto = difftime(fim,inicio)/CLOCKS_PER_SEC;  
    printf("\ntempo gasto: %f segundos\n", gasto);
}

int main () {

    FILE *x;   
    char c,arq[100];
    int i,op,co;    
    int qtd; //não é uma boa prática o uso de variáveis globais                    

    printf("Digite o nome do arquivo: ");
    scanf("%s", arq);//alterei para não usar fgets
    system("cls");

    x = fopen(arq,"r"); //deves sempre testar se não veio NULL o x
    for (c = getc(x); c != EOF; c = getc(x))
        if (c == '\n') 
            qtd = qtd + 1;
    fclose(x);

    int p[qtd]; 
    x = fopen(arq,"r"); //deveria testar se não é NULL o valor de x        
    rewind(x); //se o objetivo é apenas voltar ao início, usa o rewind

    while(x != NULL && !feof(x)){ //o for interno estava errado
          fscanf(x,"%d",&p[i]); //desta forma, ele irá ler todos
          i++;
    }

    fclose(x);  
    clock_t inicio = clock();

    do{
        printf("Menu\n");
        printf("1. Bubblesort\n");
        printf("2. Mergesort\n");
        printf("3. Quicksort\n");
        scanf("%d",&op);
        system("cls");
        clock_t fim = clock();
        double gasto = difftime(fim,inicio)/CLOCKS_PER_SEC;  
        int aux, k, j;    

        switch(op){    
        case 1:
            for(k=qtd-2;k>=0;k--){
              for(j=0;j<=k;j++){
                if(p[j]>p[j+1] ){
                   aux=p[j];
                   p[j]=p[j+1];
                   p[j+1]=aux;
                }
              }
            }       
            printf("Vetor ordenado: \n");
            for(k=0;k<qtd;k++){
                printf("%d\n",p[k]);
            } 
            printf("\ntempo gasto: %f segundos\n", gasto);       
            break;
        case 2:
            mergesort(p,0,qtd-1);   
            printf("\nVetor Ordenado: \n");
            for(i=0;i<qtd;i++){
                printf("%d\n",p[i]);
            }
            break;
        case 3:
            quicksort(p,qtd);
            printf("Vetor ordenado: \n");
            for (co = 0; co < qtd; co++) {
                printf("%d\n", p[co]);
            }                       
            break;
        default:
            printf("Entrada invalida!\n");
            printf("\ntempo gasto: %f segundos\n", gasto);                 
        } //retirei o return 0 de dentro do switch
        system("pause"); //aqui vão as instruções em comum dos cases
        system("cls");
    }while(op != 4);
    return 0;
}