Week Seventeen

December 21, 2017

We met on Tuesday for our last meeting of 2017. We began by presenting our code and results as requested from last week. Erick's code is able to go through all of our current transcription files and determine if it is usable data based on the child's age, the speaker tags (must have MOT, FAT, or PAR in the file), and the presence (or absence) of time stamps. My code is able to go through files that are considered usable data to scrape lines and time stamps from the speakers we care about and pipe them to a new file. Together, we have code that can identify and format our usable data. Below is a portion of my project (the code to scrape the files) and a sample output file.

​

public void Scrape()
        {
            DirectoryInfo outputdir = new DirectoryInfo(OutputPath);
            DirectoryInfo data = new DirectoryInfo(InputPath);

            foreach (DirectoryInfo corpra in data.GetDirectories())
            {
                foreach(FileInfo childFile in corpra.GetFiles())
                {
                    string curoutpath = OutputPath + "\\" + corpra.Name + "_" + childFile.Name;
                    if (System.IO.File.Exists(curoutpath)) System.IO.File.Delete(curoutpath);

                    System.IO.File.Create(curoutpath);
                    StreamWriter writer = new StreamWriter(curoutpath);

                    string filepath = InputPath + "\\" + corpra.Name + "\\" + childFile.Name;
                    StreamReader reader = new StreamReader(filepath);
                    string line = "";
                    while((line = reader.ReadLine()) != null)
                    {
                        if (line.Contains("*MOT:") || line.Contains("*FAT:") || line.Contains("*PAR:"))
                        {
                            line = line.Substring(6);
                            writer.WriteLine(line);
                        }
                    }

                    writer.Close();
                    reader.Close();
                }
            }
        }

​

​

what's the matter ? 3146391_3147224
that's a much better idea xxx . 3147224_3160166
he talks to himself . 3353635_3356973
what are you looking at [!!] ? 3534418_3537573
at ? 3537573_3538593
what's that Peter ? 3538593_3549079
what does she have in her hand ? 3549079_3550229
frisbee [>] . 3552151_3553168
no more pennies (.) you don't have any in your pocket (.) all_gone
where's the lady ? 3710723_3711473
is she gone ? 3711473_3712392
<what happened to the lady> [<] ? 3713487_3715173
are you playing with my feet ? 3747550_3748550
seat ? 4135129_4136162
get out ? 4203381_4204031
down ? 4204031_4205067
down ? 4206436_4207069
should we wipe your mouth first ? 4207069_4209325
not your tongue (.) your mouth (.) you gonna stick out your tongue
wash this out . 4217406_4221594
what ? 4222614_4223280
there's no more milk . 4223280_4224700
why don't you let me wash out the cup (.) do you want to wash out
cup (.) wash out the cup ? 4231997_4234869
here (.) let me wash out the cup . 4234869_4245389
there is no more (.) want some juice ? 4255558_4257510
milk is yyy . 4258227_4259413
do you want some juice now ? 4267230_4292959
xxx . 4294259_4305014
yeah [>] . 4305747_4306313
what ? 4307616_4308132
what ? 4308132_4309568
(.) <you finished> [>] ? 4309568_4311520
you want more ? 4312636_4313939
don't spill it (..) you want more ? 4317111_4324171
put your cup down (.) do you want more ? 4325104_4327806
do you want more ? 4327806_4328456
put your cup down . 4328856_4329972
I'm not gonna pour it there (be)cause I'll spill it . 4329972_4333414
put your cup down . 4333414_4337402
I'll pour you some juice (.) put it down . 4337402_4339621
oh (.) you want (.) to put that back in (.) oh I see . 4339621_4342860
what ? 4345746_4346549

​

​

The code is just a portion of the GUI that I am creating to make it easier to work with the CHILDES data, and the sample is part of one transcript for one session with a child. The lines are everything spoken by the child's parents and are our child directed speech. These data will become a semantic network and a phonological network and part of a training set for Erick's speech recognition model. 

Now that we have our usable data we can start designing models and creating algorithms that will serve as statistical tests to analyze our data. Our first step will be to create our models. For Erick that means training his system with the data sets, and for me that means creating networks for the data sets. The next step will be creating our null model (or starting our null simulation). 

The null model will be built with the following steps.

1) take an observation from our data (a network/ network measure)

2) find a new population from a normalized source (our VOX data representing normal adult speech)

     repeat this 1000x to create a normal distribution

3) is our observation unique compared to our distribution?

​

We will be able to do this multiple times and focus on how we can change our normalized model to better represent our observations. This simulation will most likely be the bulk of our papers, but later we can also separated our observations into the three time ranges (0-12, 12-24, 24-36) to see how the model changes over time. So, goals for the rest of the year and January is to finish the parser GUI to build any network we may need, and then to use R to analyze the networks and our null model, and finally to write and submit a paper to CogSci.