Let’s say you have a JSON file and in it is an array of values. Each value has a few properties you need to pick from to ultimately create a file name. Here is an example using bash, jq, in2csv and xargs that can be strung together in one line to create a file based on each concatenated filename.
cat some.json \
| jq -r '.statements | map([.account.number,.endDate,.id] | join("_") + ".pdf")' \
| in2csv -f json \
| sed -n '1d;p' \
| xargs -I{} cp fubar.pdf {};
Let’s break this down into it’s individual pieces.
Sample JSON Input
{
"statements" : [ {
"id" : "60090438-83cc-433f-bbf9-8086f4da846f",
"account" : {
"number" : "801284289140212"
},
"endDate" : "2021-06-30",
...
}
...
]
}
Using JQ To Select A Few Fields to Concatenate for a File Name
The following line will select three properties/fields out of the JSON
jq -r '.statements | map([.account.number,.endDate,.id] | join("_") + ".pdf")'
It will join
them by an underscore and then concatenate a .pdf
to the end of the file name.
Convert Quoted JQ JSON into CSV via In2Csv
The jq
command will provide quoted output. Since, there is only one value per item in the array due to the concatenation, I just want to strip out the JSON quoting.
That is done with the following:
in2csv -f json
Note the use of the -f json
since this is a piped command. That will yield this warning, which we can squash next
/usr/local/Cellar/csvkit/1.0.5_1/libexec/lib/python3.8/site-packages/agate/utils.py:276: UnnamedColumnWarning: Column 0 has no name. Using "a".
Remove The First Line From the Bash Output
This piped command remove the header labeled a
by default:
sed -n '1d;p'
Creating a Dummy Placeholder File Per File Name Generated with Xargs and Cp
Lastly, I want to generate a placeholder file (touch
could be used too) per file name. This command achieves that with xargs
xargs -I{} cp fubar.pdf {};
Sample output:
-rw-r--r--@ 1 kevincoy staff 1676399 Jun 29 16:48 182362260357264_2021-03-31_81f1465b-b2d6-43d9-87bd-d6ede8188188.pdf
-rw-r--r--@ 1 kevincoy staff 1676399 Jun 29 16:48 468913841109837_2021-03-31_043af01f-bd27-42a7-b37d-c7c4b7d1275f.pdf
-rw-r--r--@ 1 kevincoy staff 1676399 Jun 29 16:48 558664362096984_2021-03-31_287c8418-98bd-4088-9645-b897c40d0ed4.pdf
-rw-r--r--@ 1 kevincoy staff 1676399 Jun 29 16:48 881961481598397_2021-03-31_14e1e791-3e2d-4275-b27a-bf49cc2999e8.pdf
-rw-r--r--@ 1 kevincoy staff 1676399 Jun 29 16:48 201869782345979_2021-03-31_a623f37a-dfe7-4748-af2e-6b45dbeb869b.pdf
-rw-r--r--@ 1 kevincoy staff 1676399 Jun 29 16:48 177636620098765_2021-03-31_4007c4c2-7263-4d58-8a8f-7b5bd6aef802.pdf
-rw-r--r--@ 1 kevincoy staff 1676399 Jun 29 16:48 812555146927128_2021-03-31_f3a06bd9-efe3-4da1-bd29-6ee93473d920.pdf
Resources:
- https://jqplay.org/s/N6TboUkELM
- https://csvkit.readthedocs.io/en/1.0.2/scripts/in2csv.html
- https://stackoverflow.com/questions/604864/print-a-file-skipping-the-first-x-lines-in-bash
- https://unix.stackexchange.com/questions/349252/cp-after-xargs-not-working
However, this is observably different in certain rare cases. For example, if an array has a “hole” in it, the missing index will create an For certain downleveling operations, TypeScript uses some helper code for operations like extending class, spreading arrays or objects, and async operations. By default, these helpers are inserted into files which use them. This can result in code duplication if the same helper is used in many different modules.