quickly combining many gpr files into one table 01-10-2014d2210

2014-08-29

quickly combining many gpr files into one table 01-10-2014d2210
combine data from many gpr files into one table
output folder of gpr files into one table
output folder of gprs into one table
output folder of gprs to tab delimited text file
output folder of gprs to tdt

see some java code below

process for quickly combining all gpr files into one table

use Linux
list all of the files in the directory
-ex: ls>all_files.txt
take these filenames into a spreadsheet program like openoffice calc in order to construct the necessary commands
use commands like "awk 'NR >= 32' test.gpr>out.txt" to remove any headers
use commands like "cut -f9 out.txt>out2.txt" to get the column you want
use commands like the following to combined all the files into columns in a table



paste out2_1.txt out2_2.txt | column -s $'\t' -t>final_output.txt


paste final_output.txt out2_3.txt | column -s $'\t' -t>final_output2.txt


paste final_output2.txt out2_4.txt | column -s $'\t' -t>final_output3.txt

^actually all of the files can be combined in one single paste command like
paste out2_1.txt out2_2.txt out2_3.txt | column -s $'\t' -t>final_output.txt

Currently it looks like I cannot combine more than 138 files into columns. I do not know how to overcome this limitation at this time. Therefore, I think I will approach this issue by combining 100 files at a time. Then I can combine tables separately by copying and pasting in excel with a good computer.

I've also made some Java code to automatically generate the necessary Linux code to combine all of the gpr files.

java code for extracting columns from many files and forming one table 01-11-2014d1418
-the linux code generated from this java program can combine 100 10k files in less than 1 min even on a very slow linux desktop running in a virtual os. Great!