Filter Rows by Numeric Threshold

Intermediate Bash Foundations Bash awk Filtering Differential Expression
Significance:

Every differential expression analysis ends with a filtering step: keep the genes whose
adjusted p-value is below a cutoff and whose fold change exceeds some magnitude. Doing that in awk
rather than loading the whole table into R is how you handle files too large to open, and it is the
fastest way to answer a quick question about a results table without leaving the terminal.

Statement

Tab-separated data arrives on standard input. The first line of input is a numeric
threshold. The second line is a header row. Every remaining line is a data row whose third column is
a numeric p-value.

Print the complete data rows — all columns, tab separated, exactly as they appeared — for which the
third column is strictly less than the threshold, preserving input order.

Do not print the header. If no rows pass, print nothing.

Input — read from standard input
Variable Type Description
threshold
line 1
float The p-value cutoff
0 < threshold <= 1
header
line 2
str The tab-separated header row, which is skipped
exactly 3 columns
rows
line 3..n
str Tab-separated data rows with a numeric third column
0 <= rows <= 1000

These variables are already read for you in the starter code on the right.

Output

str the full data rows whose third column is strictly below the threshold, in input order

Sample Cases
Sample 1
Input
0.05
gene	logFC	pval
TP53	2.5	0.001
BRCA1	-1.2	0.03
EGFR	0.4	0.5
MYC	3.1	0.049
Expected Output
TP53	2.5	0.001
BRCA1	-1.2	0.03
MYC	3.1	0.049
Three genes fall below the significance cutoff; the fourth does not.
Sample 2
Input
0.01
gene	fc	p
A	1.0	0.001
B	2.0	0.01
Expected Output
A	1.0	0.001
The second row equals the threshold exactly and is therefore excluded.

Submit also runs your code against 3 hidden test cases. Hidden inputs are never shown — if one fails you'll get its number and a description of the mismatch, not the data.

Constraints
  • Comparison is strictly less than: a row equal to the threshold is excluded
  • Rows are printed unchanged, including all their columns and original tab separators
  • The header row is never printed
  • Scientific notation such as 1e-05 may appear in the third column and must compare correctly
Further Reading
  • read -r thr pulls the threshold off the front of the stream before awk sees the rest.
  • Pass the threshold in with -v t="$thr" so awk treats it as a number.
  • $0 holds the entire unmodified line, which is what you want to print.

Judged against GNU Bash 5.1.0.


My Notes
Log in to save personal notes.
Console output will appear here when you click Run Code or Submit...
Expected: threshold (float), header (str), rows (remaining lines)
Next Problem
Count Unique Values in a Stream