Grep is an amazing tool to search through log files and pull out useful information, but what if you want to search a log file using a giant list of keywords from another file? Luckily it has that feature built in as well.
This happened to me when I wanted to pull a list of all URLs that were requested by a huge block of IPs that was abusively attacking our server. After identifying and creating the list of a thousand IPs, I needed to pull the URLs from the main log file to identify the most requested resources.
To do this you’ll want to use the -f argument, which allowes you to specify a file for the list of patterns to search for.
Assuming your set of keywords or strings is in a file named “searchstrings”, you can use the argument on the command line like the following example. Since this search is going to generate a ton of data, the “> output.txt” part of the command sends the result of the command into a file called output.txt that can be analyzed separately.
The only issue with using the -f argument is that grep is going to attempt to interpret the keywords as if they are patterns, which can slow it down when parsing against an extremely large file. So you can also specify the -F parameter, which tells grep to only do exact matches against the strings.
So the full command would end up being more like this:
Grep is a ridiculously powerful way to search log files, so it would be well worth your time to look through the man file.
