some work on entropy 330k data as of 9-15-13

2014-08-29

++ some work on entropy 330k data as of 9-15-13

I started exploring some of the entropy results.
F:\kurt\storage\CIM Research Folder\DR\2013\9-13-13\entropy

Summary spreadsheet of entropy values for the 330k data here
"F:\kurt\storage\CIM Research Folder\DR\2013\9-10-13\entropy\summary of entropy values 9-10-13.xlsx"

Here's an inkscape file with various graphs.
"F:\kurt\storage\CIM Research Folder\DR\2013\9-13-13\entropy\Graphs of 330K data 9-13-13.svg"
^make sure to find most recent version

graphs present
-Entropy wafer 22 (Classification 1)
-Entropy wafer 22 (Type)
-CV wafer 22 (Type)
-CV wafer 22 (Classification 1)
-Normalized Entropy wafer 22 (Classification 1)
-Normalized Entropy wafer 22 (Type)
-Normalized CV wafer 22 (Type)
-Entropy wafer 46 (Type)
-Entropy wafer 46 (Classification 1)
-cv wafer 46 (Type)
-cv wafer 46 (Classification 1)
-normalized entropy wafer 46 (Type)
-Normalized Entropy wafer 46 (classification 1)
-Normalized CV wafer 46 (classification 1)
-Normalized CV wafer 46 (type)

parameters for dot plot
-set group and type
-alpha at 0.5
-binwidth at 0.01
-dotsize at 3.0

some of the samples were incomplete because the saguaro supercomputer did not properly calculate the entropy or cv numbers for some of the samples (some folders were missing the gpr file, some results just had 0s or NaNs, etc.)

I considered going back and having Saguaro run all of the samples that didn't run well, but this could become complicated and time consuming so I think I'll just settle with the data that I have.
^Actually I'll go back and rerun the few good samples that I missed with wafer 46. Many of them were F635 instead of F531 (or F532 I forget) samples, and two of them were just titled red green something, and I'm not sure what that is.
Here is the key for samples that didn't run the 1st time (blank means there should be no problem)



101	F635 Median


82


83	F635 Median


85	F635


86	


87	F635


88	


89	F635


91	F635


93	F635


94	


95	F635


98	F635


99	F635


84	green red p50 sample


90	green red P50 sample

list of 12 samples to rerun



42	4-46 S4 H3 Hi P20 111312 DTRA 1 S.gpr


43	4-46 S5 A1 Hi P20 03252013 ND84 L.gpr


86	4-46 S8 B2 Hi R-P50 G-P20 03012013 Normal 154_0532 A.gpr


88	4-46 S8 B3 Hi R-P50 G-P20 03012013 GMB OOKN_0532 A.gpr


94	4-46 S8 D3 Hi R-P50 G-P20 03012013 Mening XX_0532 A.gpr


100	4-46 S8 G2 Hi R-P50 G-P20 03012013 EC 722078_0532 K.gpr


102	HT4-46 S1 B2 Hi P20 103112 #1-1 S.gpr


104	HT4-46 S1 D1 Hi P20 103112 #4-7 K.gpr


105	HT4-46 S1 E1 Hi P20 103112 BC040 K.gpr


106	HT4-46 S1 F1 Hi P20 103112 ND145 1-50k L.gpr


108	HT4-46 S1 G2 Hi P20 103112 DM1A 500.0 pM S.gpr


82	4-46 S8 A3 Hi R-P50 G-P20 03012013 EC 712707_0532 K.gpr

In order to start this job, I need to modify the cmd_original.saguaro file from Josh.

The path to the files is now:
/home/lwang138/entropy/input/rerun_of_46

Example of one command:
java -jar "/home/lwang138/entropy/input/EntropyOfArray090413_2.jar" /home/lwang138/entropy/input/rerun_of_46/42" "4-46 S4 H3 Hi P20 111312 DTRA 1 S.gpr" -Xmx6144

What were some of the last commands that were in the terminal?

qstat -n
mybalance
cd /home/lwang138/kurt
chmod +x jobScript.saguaro
./jobScript.saguaro cmd.saguaro pbs.saguaro

Here are the commands I will issue now to run these remaining files.

cd /home/lwang138/kurt/2nd_run_9-14-13
chmod +x jobScript.saguaro
./jobSc-ript.saguaro cmd_original.saguaro pbs.saguaro

There was some problem when I tried to run the program.

Here is the list of jobs



6154356.newmoab            jobTemp.saguaro  lwang138        00:00:00 C serial         


6154357.newmoab            jobTemp.saguaro  lwang138        00:00:00 C serial         


6154358.newmoab            jobTemp.saguaro  lwang138        00:00:00 C serial         


6154359.newmoab            jobTemp.saguaro  lwang138        00:00:00 C serial         


6154360.newmoab            jobTemp.saguaro  lwang138        00:00:00 C serial         


6154361.newmoab            jobTemp.saguaro  lwang138        00:00:00 C serial         


6154362.newmoab            jobTemp.saguaro  lwang138        00:00:14 C serial         


6154363.newmoab            jobTemp.saguaro  lwang138        00:00:14 C serial         


6154364.newmoab            jobTemp.saguaro  lwang138        00:00:00 C serial         


6154365.newmoab            jobTemp.saguaro  lwang138        00:00:00 C serial         


6154366.newmoab            jobTemp.saguaro  lwang138        00:00:00 C serial

The message for the first job is this:
/var/spool/torque/mom_priv/jobs/6154356.newmoab.local.SC: line 7: unexpected EOF while looking for matching `"'
/var/spool/torque/mom_priv/jobs/6154356.newmoab.local.SC: line 8: syntax error: unexpected end of file

The command looked like this:
java -jar "/home/lwang138/entropy/input/EntropyOfArray090413_2.jar" /home/lwang138/entropy/input/rerun_of_46/42" "4-46 S4 H3 Hi P20 111312 DTRA 1 S.gpr" -Xmx6144
^It looks like I had an extra quotation that I did not need.

I tried running again, but I ran into another problem. I forgot that .gpr should not be a part of the filename. I deleted the .gpr from the commands and tried running again.