Friday, May 17, 2024
0
rated 0 times [  0] [ 0]  / answers: 1 / hits: 881  / 2 Years ago, mon, june 13, 2022, 7:30:50

Is Ubuntu different from Linux on a Mac? Sorry if this is a basic question...I am new to this. I am trying to run a bash script that creates a .sh script out of some FASTQ files. This works in the terminal on a Mac OS. I am trying to run it on my Windows laptop and it ignores my escaping of #s and just states that several commands are not found. I have tried using dos2unix and double checked with cat -A file.sh but it hasn't helped.


The code I am trying to run takes all fastq files in a folder and creates a .sh file for SLURM job submissions using their file name (needed for my university's computer cluster, and I need to make 100+ job scripts). So the Mac OS version is as follows:


for FILE in *fastq;    #change file type when needed (e.g., fasta, fastq, fastq.gz)
do echo -e
#!/bin/bash
n#SBATCH --partition=nonpre # Partition (job queue)
n#SBATCH --requeue # Return job to the queue if preempted
n#SBATCH --job-name=samples # Assign a short name to your job
n#SBATCH --nodes=1 # Number of nodes you require
n#SBATCH --ntasks=1 # Total # of tasks across all nodes
n#SBATCH --cpus-per-task=64 # Cores per task (>1 if multithread tasks)
n#SBATCH --mem=180000 # Real memory (RAM) required (MB)
n#SBATCH --time=72:00:00 # Total run time limit (HH:MM:SS)
n#SBATCH --output=slurm.%N.${FILE}.out # STDOUT output file
n#SBATCH --error=slurm.%N.${FILE}.err # STDERR output file (optional)
n
n#ADD WHATEVER CODE YOU WANT HERE AS YOUR SLURM JOB SUBMISSION
n
nsacct --format=JobID,JobName,NTasks,NNodes,NCPUS,MaxRSS,AveRSS,AveCPU,Elapsed,ExitCode -j $SLURM_JOBID #this will get job run stats from SLURM; use these to help designate memory of future submissions
> ${FILE}.sh;
done

When running this on Windows, I get:


Slurm_Generator.sh: line 13: #!/bin/bash: No such file or directory
Slurm_Generator.sh: line 14:
#SBATCH: command not found
Slurm_Generator.sh: line 16:
#SBATCH: command not found
Slurm_Generator.sh: line 18:
#SBATCH: command not found
Slurm_Generator.sh: line 20:
#SBATCH: command not found
Slurm_Generator.sh: line 22:
#SBATCH: command not found
Slurm_Generator.sh: line 24:
#SBATCH: command not found
Slurm_Generator.sh: line 26:
#SBATCH: command not found
Slurm_Generator.sh: line 28:
#SBATCH: command not found
Slurm_Generator.sh: line 30:
#SBATCH: command not found
Slurm_Generator.sh: line 32:
#SBATCH: command not found
Slurm_Generator.sh: line 35:
: command not found
Slurm_Generator.sh: line 36:
#ADD: command not found
Slurm_Generator.sh: line 39:
: command not found
Slurm_Generator.sh: line 40:
sacct: command not found

Any help would be appreciated, and some explanation on what the difference is between Ubuntu on Windows vs. the Terminal on Mac. I've tried researching this but I keep finding suggested code with no explanation or it isn't particularly my issue. Thank you!


Edit: I tried running chmod +x script.sh and I will get the above errors. Am I running 'echo' wrong? Even running:
'for FILE in *fastq;
do
echo -e
hello;
done'
says Command 'hello' not found


Edit: Running 'bash file.sh' yields the following:
bash file.sh yields the following (for each of the 5 .fastq files in my directoy):


Slurm_Generator.sh: line 8: #!/bin/bash: No such file or directory
Slurm_Generator.sh: line 9:
#SBATCH: command not found
Slurm_Generator.sh: line 11:
#SBATCH: command not found
Slurm_Generator.sh: line 13:
#SBATCH: command not found
Slurm_Generator.sh: line 15:
#SBATCH: command not found
Slurm_Generator.sh: line 17:
#SBATCH: command not found
Slurm_Generator.sh: line 19:
#SBATCH: command not found
Slurm_Generator.sh: line 21:
#SBATCH: command not found
Slurm_Generator.sh: line 23:
#SBATCH: command not found
Slurm_Generator.sh: line 25:
#SBATCH: command not found
Slurm_Generator.sh: line 27:
#SBATCH: command not found
Slurm_Generator.sh: line 30:
: command not found
Slurm_Generator.sh: line 31:
#ADD: command not found
Slurm_Generator.sh: line 34:
: command not found
Slurm_Generator.sh: line 35:
sacct: command not found

If I run cat -A file.sh I see a $ at the end of each line. Even if I get rid of these, I get the same result as above.
Running ls -al script.sh gived: -rwxrwxrwx 1 cerberus cerberus 1209 Jun 16 00:17 Slurm_Generator.sh


Edit:
I changed my script to:


#! /bin/bash
for FILE in *fastq; #change file type when needed (e.g., fasta, fastq, fastq.gz)
do echo -e
"
#!/bin/bash
#SBATCH --partition=nonpre # Partition (job queue)
#SBATCH --requeue # Return job to the queue if preempted
#SBATCH --job-name=samples # Assign a short name to your job
#SBATCH --nodes=1 # Number of nodes you require
#SBATCH --ntasks=1 # Total # of tasks across all nodes
#SBATCH --cpus-per-task=64 # Cores per task (>1 if multithread tasks)
#SBATCH --mem=180000 # Real memory (RAM) required (MB)
#SBATCH --time=72:00:00 # Total run time limit (HH:MM:SS)
#SBATCH --output=slurm.%N.${FILE}.out # STDOUT output file
#SBATCH --error=slurm.%N.${FILE}.err # STDERR output file (optional)

#ADD WHATEVER CODE YOU WANT HERE AS YOUR SLURM JOB SUBMISSION

sacct --format=JobID,JobName,NTasks,NNodes,NCPUS,MaxRSS,AveRSS,AveCPU,Elapsed,ExitCode -j $SLURM_JOBID #this will get job run stats from SLURM; use these to help designate memory of future subm$"
> ${FILE}.sh;
done

My new output is the following (which is much better):


#!/bin/bash
#SBATCH --partition=nonpre # Partition (job queue)
#SBATCH --requeue # Return job to the queue if preempted
#SBATCH --job-name=samples # Assign a short name to your job
#SBATCH --nodes=1 # Number of nodes you require
#SBATCH --ntasks=1 # Total # of tasks across all nodes
#SBATCH --cpus-per-task=64 # Cores per task (>1 if multithread tasks)
#SBATCH --mem=180000 # Real memory (RAM) required (MB)
#SBATCH --time=72:00:00 # Total run time limit (HH:MM:SS)
#SBATCH --output=slurm.%N.Sample1.fastq.out # STDOUT output file
#SBATCH --error=slurm.%N.Sample1.fastq.er # STDERR output file (optional)


#ADD WHATEVER CODE YOU WANT HERE AS YOUR SLURM JOB SUBMISSION

sacct --format=JobID,JobName,NTasks,NNodes,NCPUS,MaxRSS,AveRSS,AveCPU,Elapsed,ExitCode -j $SLURM_JOBID #this will get job run stats from SLURM; use these to help designate memory of future submissions

The only issue I am running into is that while it prints this out for each .fastq file, which is what I want, the resulting .sh files that it writes out are blank. So it is not recognizing the > ${File}.sh part of the script.


Thank you everyone!


More From » command-line

 Answers
3

I realized that I had to add a before the > ${File}.sh. So the answer is:


#! /bin/bash
for FILE in *fastq; #change file type when needed (e.g., fasta, fastq, fastq.gz)
do echo -e
"
#!/bin/bash
#SBATCH --partition=nonpre # Partition (job queue)
#SBATCH --requeue # Return job to the queue if preempted
#SBATCH --job-name=samples # Assign a short name to your job
#SBATCH --nodes=1 # Number of nodes you require
#SBATCH --ntasks=1 # Total # of tasks across all nodes
#SBATCH --cpus-per-task=64 # Cores per task (>1 if multithread tasks)
#SBATCH --mem=180000 # Real memory (RAM) required (MB)
#SBATCH --time=72:00:00 # Total run time limit (HH:MM:SS)
#SBATCH --output=slurm.%N.${FILE}.out # STDOUT output file
#SBATCH --error=slurm.%N.${FILE}.err # STDERR output file (optional)

#ADD WHATEVER CODE YOU WANT HERE AS YOUR SLURM JOB SUBMISSION

sacct --format=JobID,JobName,NTasks,NNodes,NCPUS,MaxRSS,AveRSS,AveCPU,Elapsed,ExitCode -j $SLURM_JOBID #this will get job run stats from SLURM; use these to help designate memory of future subm$"
> ${FILE}.sh;
done

[#1444] Monday, June 13, 2022, 2 Years  [reply] [flag answer]
Only authorized users can answer the question. Please sign in first, or register a free account.
fertion

Total Points: 436
Total Questions: 121
Total Answers: 156

Location: England
Member since Sun, May 21, 2023
1 Year ago
fertion questions
Thu, Mar 9, 23, 06:46, 1 Year ago
Thu, Jun 17, 21, 01:06, 3 Years ago
Fri, Mar 10, 23, 12:13, 1 Year ago
Thu, Jan 6, 22, 22:08, 2 Years ago
;