Is Ubuntu different from Linux on a Mac? Sorry if this is a basic question...I am new to this. I am trying to run a bash script that creates a .sh script out of some FASTQ files. This works in the terminal on a Mac OS. I am trying to run it on my Windows laptop and it ignores my escaping of #s and just states that several commands are not found. I have tried using dos2unix
and double checked with cat -A file.sh
but it hasn't helped.
The code I am trying to run takes all fastq files in a folder and creates a .sh file for SLURM job submissions using their file name (needed for my university's computer cluster, and I need to make 100+ job scripts). So the Mac OS version is as follows:
for FILE in *fastq; #change file type when needed (e.g., fasta, fastq, fastq.gz)
do echo -e
#!/bin/bash
n#SBATCH --partition=nonpre # Partition (job queue)
n#SBATCH --requeue # Return job to the queue if preempted
n#SBATCH --job-name=samples # Assign a short name to your job
n#SBATCH --nodes=1 # Number of nodes you require
n#SBATCH --ntasks=1 # Total # of tasks across all nodes
n#SBATCH --cpus-per-task=64 # Cores per task (>1 if multithread tasks)
n#SBATCH --mem=180000 # Real memory (RAM) required (MB)
n#SBATCH --time=72:00:00 # Total run time limit (HH:MM:SS)
n#SBATCH --output=slurm.%N.${FILE}.out # STDOUT output file
n#SBATCH --error=slurm.%N.${FILE}.err # STDERR output file (optional)
n
n#ADD WHATEVER CODE YOU WANT HERE AS YOUR SLURM JOB SUBMISSION
n
nsacct --format=JobID,JobName,NTasks,NNodes,NCPUS,MaxRSS,AveRSS,AveCPU,Elapsed,ExitCode -j $SLURM_JOBID #this will get job run stats from SLURM; use these to help designate memory of future submissions
> ${FILE}.sh;
done
When running this on Windows, I get:
Slurm_Generator.sh: line 13: #!/bin/bash: No such file or directory
Slurm_Generator.sh: line 14:
#SBATCH: command not found
Slurm_Generator.sh: line 16:
#SBATCH: command not found
Slurm_Generator.sh: line 18:
#SBATCH: command not found
Slurm_Generator.sh: line 20:
#SBATCH: command not found
Slurm_Generator.sh: line 22:
#SBATCH: command not found
Slurm_Generator.sh: line 24:
#SBATCH: command not found
Slurm_Generator.sh: line 26:
#SBATCH: command not found
Slurm_Generator.sh: line 28:
#SBATCH: command not found
Slurm_Generator.sh: line 30:
#SBATCH: command not found
Slurm_Generator.sh: line 32:
#SBATCH: command not found
Slurm_Generator.sh: line 35:
: command not found
Slurm_Generator.sh: line 36:
#ADD: command not found
Slurm_Generator.sh: line 39:
: command not found
Slurm_Generator.sh: line 40:
sacct: command not found
Any help would be appreciated, and some explanation on what the difference is between Ubuntu on Windows vs. the Terminal on Mac. I've tried researching this but I keep finding suggested code with no explanation or it isn't particularly my issue. Thank you!
Edit: I tried running chmod +x script.sh
and I will get the above errors. Am I running 'echo' wrong? Even running:
'for FILE in *fastq;
do
echo -e
hello;
done'
says Command 'hello' not found
Edit: Running 'bash file.sh' yields the following:
bash file.sh
yields the following (for each of the 5 .fastq files in my directoy):
Slurm_Generator.sh: line 8: #!/bin/bash: No such file or directory
Slurm_Generator.sh: line 9:
#SBATCH: command not found
Slurm_Generator.sh: line 11:
#SBATCH: command not found
Slurm_Generator.sh: line 13:
#SBATCH: command not found
Slurm_Generator.sh: line 15:
#SBATCH: command not found
Slurm_Generator.sh: line 17:
#SBATCH: command not found
Slurm_Generator.sh: line 19:
#SBATCH: command not found
Slurm_Generator.sh: line 21:
#SBATCH: command not found
Slurm_Generator.sh: line 23:
#SBATCH: command not found
Slurm_Generator.sh: line 25:
#SBATCH: command not found
Slurm_Generator.sh: line 27:
#SBATCH: command not found
Slurm_Generator.sh: line 30:
: command not found
Slurm_Generator.sh: line 31:
#ADD: command not found
Slurm_Generator.sh: line 34:
: command not found
Slurm_Generator.sh: line 35:
sacct: command not found
If I run cat -A file.sh
I see a $
at the end of each line. Even if I get rid of these, I get the same result as above.
Running ls -al script.sh
gived: -rwxrwxrwx 1 cerberus cerberus 1209 Jun 16 00:17 Slurm_Generator.sh
Edit:
I changed my script to:
#! /bin/bash
for FILE in *fastq; #change file type when needed (e.g., fasta, fastq, fastq.gz)
do echo -e
"
#!/bin/bash
#SBATCH --partition=nonpre # Partition (job queue)
#SBATCH --requeue # Return job to the queue if preempted
#SBATCH --job-name=samples # Assign a short name to your job
#SBATCH --nodes=1 # Number of nodes you require
#SBATCH --ntasks=1 # Total # of tasks across all nodes
#SBATCH --cpus-per-task=64 # Cores per task (>1 if multithread tasks)
#SBATCH --mem=180000 # Real memory (RAM) required (MB)
#SBATCH --time=72:00:00 # Total run time limit (HH:MM:SS)
#SBATCH --output=slurm.%N.${FILE}.out # STDOUT output file
#SBATCH --error=slurm.%N.${FILE}.err # STDERR output file (optional)
#ADD WHATEVER CODE YOU WANT HERE AS YOUR SLURM JOB SUBMISSION
sacct --format=JobID,JobName,NTasks,NNodes,NCPUS,MaxRSS,AveRSS,AveCPU,Elapsed,ExitCode -j $SLURM_JOBID #this will get job run stats from SLURM; use these to help designate memory of future subm$"
> ${FILE}.sh;
done
My new output is the following (which is much better):
#!/bin/bash
#SBATCH --partition=nonpre # Partition (job queue)
#SBATCH --requeue # Return job to the queue if preempted
#SBATCH --job-name=samples # Assign a short name to your job
#SBATCH --nodes=1 # Number of nodes you require
#SBATCH --ntasks=1 # Total # of tasks across all nodes
#SBATCH --cpus-per-task=64 # Cores per task (>1 if multithread tasks)
#SBATCH --mem=180000 # Real memory (RAM) required (MB)
#SBATCH --time=72:00:00 # Total run time limit (HH:MM:SS)
#SBATCH --output=slurm.%N.Sample1.fastq.out # STDOUT output file
#SBATCH --error=slurm.%N.Sample1.fastq.er # STDERR output file (optional)
#ADD WHATEVER CODE YOU WANT HERE AS YOUR SLURM JOB SUBMISSION
sacct --format=JobID,JobName,NTasks,NNodes,NCPUS,MaxRSS,AveRSS,AveCPU,Elapsed,ExitCode -j $SLURM_JOBID #this will get job run stats from SLURM; use these to help designate memory of future submissions
The only issue I am running into is that while it prints this out for each .fastq file, which is what I want, the resulting .sh files that it writes out are blank. So it is not recognizing the > ${File}.sh
part of the script.
Thank you everyone!