• Posted on

In this bash tutorial we will check out recursive grep usage examples and how it can be extremely beneficial in some use cases.

Table of Contents

  • Basics of grep -R
    1. Excluding Directories
    2. How to remove “Permission denied” messages
    3. Ignoring case sensitivity
    4. You can combine grep with regex
  • Benefits of recursive grep

grep -R, Recursive Query Function

grep -R can be used for recursive text search across many files in a directory. It can be extremely useful and can be applied to many different application cases.

1. Excluding Directories

Sometimes specific folders can cloud the query results when using grep recursively. For example you might have specific folders full of the stuff you are searching for but you might find to look for your query everywhere else under the same structure.
A typical occurrence is when you search under the root folder and proc directory becomes a nuisance in the results. This folder includes process IDs and cpu and kernel specific information which is often irrelevant for search results. Here is an example for excluding proc directory when using grep recursively but you can use it for any directory you like.
grep -R --eclude-dir=/proc "import requests" /

grep recursively looks for “import requests” under / (root) while excluding /proc directory.

2. How to remove "Permission denied" messages

grep -R  "import" /

Sometimes when you might have to use grep without sudo. Or some system directories might be unreachable even for the admin. In those cases you might get results with hundreds of lines similar to below which will be quite hard to read.

Check out an excerpt of the results from the bash script above.

/usr/share/doc/libjpeg-turbo/TJExample.java:import java.io.*;
grep: /proc/net/ip_tables_names: Permission denied
grep: /proc/net/ip_tables_matches: Permission denied
grep: /proc/net/ip_tables_targets: Permission denied
grep: /proc/sys/fs/binfmt_misc/register: Permission denied
grep: /proc/sys/kernel/cad_pid: Permission denied
grep: /proc/sys/kernel/usermodehelper/bset: Permission denied
grep: /proc/sys/kernel/usermodehelper/inheritable: Permission denied
grep: /proc/sys/net/core/bpf_jit_harden: Permission denied
grep: /proc/sys/net/core/bpf_jit_kallsyms: Permission denied
grep: /proc/sys/net/core/bpf_jit_limit: Permission denied
grep: /proc/sys/net/ipv4/route/flush: Permission denied
grep: /proc/sys/net/ipv4/tcp_fastopen_key: Permission denied
grep: /proc/sys/net/ipv6/conf/all/stable_secret: Permission denied
grep: /proc/sys/net/ipv6/conf/default/stable_secret: Permission denied
grep: /proc/sys/net/ipv6/conf/lo/stable_secret: Permission denied
grep: /proc/sys/net/ipv6/conf/wlp58s0/stable_secret: Permission denied
grep: /proc/sys/net/ipv6/route/flush: Permission denied
grep: /proc/sys/vm/compact_memory: Permission denied
grep: /proc/sys/vm/drop_caches: Permission denied
grep: /proc/sys/vm/mmap_rnd_bits: Permission denied
grep: /proc/sys/vm/mmap_rnd_compat_bits: Permission denied
grep: /proc/sys/vm/stat_refresh: Permission denied
grep: /proc/tty/driver: Permission denied
grep: /proc/kmsg: Permission denied
grep: /proc/mtrr: Operation not permitted
grep: /proc/kcore: Permission deniedgrep: /proc/slabinfo: Permission denied
grep: /proc/kpagecount: Permission denied
grep: /proc/kpageflags: Permission denied
grep: /proc/timer_list: Permission denied
grep: /proc/kpagecgroup: Permission denied
grep: /proc/vmallocinfo: Permission denied
grep: /proc/pagetypeinfo: Permission denied
grep: /proc/sysrq-trigger: Permission denied
Permission denied output can be excluded by adding 2> /dev/null to the query. Here is an example. 
grep -R  "import requests" /home/usa 2> /dev/null
/usr/share/doc/libjpeg-turbo/TJExample.java:import java.io.*;

2. Ignoring case sensitivity

grep -R  --ignore-case "IMPORT" /

You can also disable case sensitivity when using grep as above.

3. You can combine grep with regex

Regex is beyond the scope of this tutorial but you can combine grep to create very specifically tailored queries. This can be useful in server administration, cloud migration, cloud management, devops operations and similar industry-level tasks.

grep -R  --ignore-case "IMPORT" /

Benefits of Recursive Grep

Using grep recursively can be incredibly useful. Here are a few examples.

  • You can find files without knowing the file names. Imagine you recovered some data but file names are a mess. Let’s say you have 100K files with weird names. You could use grep -R for recursion and find specific files with specific content. For example you can search for a query like import requests or another Python code and you will get Python. files.
  • You can find specific text strings across many text files. You can search for an Einstein quote or a price information without opening each text file.
  • Applying a similar logic you can filter html files or you can query images. Most images include EXIF information which can be used to find specific images as well.

grep -R will treat each file as strings and go through every single file in the specified folder and output query results.

References

[1] Grep Command: GNU Grep Manual

Umut is a finance and tech professional based in London.

Recommended Posts

Solution to os-prober not finding other operating systems (Windows & Linux)
Fixing Grub on an Encrypted Luks Disc
How to Properly Serve Python Apps with PM2 and Gunicorn on any Port and any Server