What you get after installation with this custom install profile:

• Drupal 7 core
• Core + selected module(s) installed and enabled
• A custom user role and user name passed in on the command-line

Server Setup:

For a testing sandbox it will be easiest to use a Linux box or VM that has the following installed:

• Apache or lighttp w/ php and sqlite
• php w/ sqlite
• Drush v5.x

Use your OS’s packaging system to install the webserver with php and database support.

Make sure you drush is version 5.x, to install it using php pear:

pear channel-discover pear.drush.org pear install drush/drush

Or see http://drupal.org/project/drush

Drush Make

The command “drush make” will read a makefile that has some directives to tell drush where to fetch the core, modules and themes. Here is the drush makefile that we will use that installs the core plus the Role Delegation module.

; Core Drupal ; ------------- core = 7.x api = 2 projects[drupal][version] = 7 ; Modules ; ------------- projects[role_delegation][subdir] = contrib

You can tell drush where to fetch the drupal core and the module but in this case it will get it from drupal.org. There are more powerful things you can do with the makefile, see the docs for more info.

To use this makefile and Drush to install Drupal into a web directory:

drush make /path/to/makefile /path/to/install/dir

This will download and explode the Drupal core and the module into a directory which will then allow you to run the install wizard by pointing a browser to it. What you most likely want however is to automatically install Drupal and get everything set up in one shot. If all you want to do is install the core and an admin user then you can stop reading here; use the drupal site-install command which has command-line arguments for the basic options.

Read on if you want to learn how to make additional customizations.

Drupal Install Profiles

The standard install profile is the default one that is used for a vanilla Drupal core installation. Using that profile as a starting point you can start making customizations like adding modules to the default list and you can add your own install steps. This example will create an additional user and permission role called “Content Creator.” This user will have the abilities to create new article content and some admin abilities like changing themes.

Copy the standard profile into a new one called my_profile

cd /path/to/install/dir cp -r profiles/standard profiles/my_profile

In the “my_profile” directory, rename all references to the name “standard” to “my_profile.”

my_profile.info

The info file has key=value pairs that informs the profile which modules will be activated, add the “role_delegation” module to the list. This is necessary to activate the module on install.

... dependencies[] = role_delegation

my_profile.profile

This file can be customized to add additional install steps. We are going to add a step to ask for a username and password for an additional user who will be given the custom “content creater” user role. Add the following php code to the end of my_profile.profile.

... /** * Implements hook_install_tasks(). */ function my_profile_install_tasks() {$tasks = array(); // Add a page allowing the user to specify a "content creator" user $tasks['my_profile_cc_form'] = array( 'display_name' => st('Content creator username'), 'type' => 'form', ); return $tasks; }/** * Task callback: returns the form allowing the user to add * a "content creator" user */ function my_profile_cc_form() {drupal_set_title(st('Content Creator Username')); $form['cc_uid'] = array( '#type' => 'textfield', '#title' => st('Username for Content Creator:'), '#description' => st('Enter the content creator userid'), ); $form['cc_email'] = array( '#type' => 'textfield', '#title' => st('Email for Content Creator:'), '#description' => st('Enter the content creator email'), ); $form['cc_pass'] = array( '#type' => 'textfield', '#title' => st('Password for Content Creator:'), '#description' => st('Enter the content creator password in both fields'), ); $form['actions'] = array('#type' => 'actions'); $form['actions']['submit'] = array( '#type' => 'submit', '#value' => st('Create content creator role and user'), '#weight' => 15, ); return $form; }/** * Submit callback: creates the "content creator" role and user */ function my_profile_cc_form_submit(&$form, &$form_state) {$uid = $form_state['values']['cc_uid']; $email = $form_state['values']['cc_email']; $pass = $form_state['values']['cc_pass']; // Create a role for "content managers" $c_role = new stdClass(); $c_role->name = 'content manager'; user_role_save($c_role); // additional permissions beyond what the authenticated // user receives user_role_grant_permissions($c_role->rid, array( 'assign content manager role', 'create article content', 'edit own article content', 'delete own article content', 'create page content', 'edit own page content', 'delete own page content', 'administer themes', )); $cc_user = array ('name' => $pass, 'pass' => $pass, 'roles' => array($c_role->rid => $c_role->rid), 'mail' => $email, 'status' => 1, # status: active ); $user = user_save(NULL, $cc_user); }

• my_profile_install_tasks() – adds the additional install task
• my_profile_cc_form() – specifies the custom form to get user input
• my_profile_cc_form_submit() – runs when the form is submitted, creates a custom role with limited permissions and creates the user with this role assigned.

my_profile.install

This file has php code that will run on installation, it sets up the basic views, sets the theme, etc. This can be changed or extended in whatever way you want but for this example we will stick with the standard setup.

Once this profile is created it will be available as a new option on the install wizard:

And the new custom screen to create the “content creator” user:

There are many different form elements and attributes that you can read about in the Drupal form api documentation. For this example it would probably be better to use a password_confirm text input for the password but since the goal is to automate it from the commandline it doesn’t really matter.

Automating everything from the command-line

Now that the profile is customized instead of running through the wizard it would be nice to input everything from the command-line. To do this the drush site-install gives you the option of passing form parameters into the command line.

Here is what you would do to automate everything from the command-line where shell variables correspond to the “CC” (content creator) user information and admin account info.

(create the makefile) drush make /path/to/makefile /path/to/install/dir (create the new profile) cd /path/to/install/dir drush -y site-install --clean-url=0 --db-url=sqlite:sites/default/files/db.sqlite --account-name=$ADMIN_USER --account-pass=$ADMIN_PASS --account-mail=$ADMIN_MAIL --site-mail=$SITE_MAIL my_profile my_profile_cc_form.cc_uid=$CC_USER my_profile_cc_form.cc_email=$CC_MAIL my_profile_cc_form.cc_pass=$CC_PASS

“my_profile_cc_form” is the name of the form for the custom “Content Creator” install step and cc_uid, cc_email, and cc_pass are the parameters that entered. When that command completes you will have a fully functional site with a custom user and role.

If you something isn’t working for you I have put all of the above into a standalone shell script, simply install drush and and change the variable assignments at the top to suite your needs.

This post takes a look at supreme court transcripts as corpus for natural language processing. Lately I’ve been playing around with the nltk python module and I thought this might be an interesting data set (given that I’m also an avid follower of SCOTUS).

Supreme Court transcripts are made available by supremecourt.gov and can be downloaded in PDF form. Extracting data from the PDFs is not an exact science since the format varies a bit. I can get a majority of the cases broken up by speaker for what is available for download. The script does its best to compensate for transcription errors, typos, etc. Nothing is perfect though so it’s only useful to look at this data in the aggregate.

Click here for transcripts by person organized by name, or here for organized by case.

Stats from all available transcript data

“Other speakers” are petitioners and respondents (not justices)
“stopwords” are high frequency words like the, to, also, etc.
“words”, “unique words” do not include “stopwords”

Stats from 500 randomly selected statements

Justice Thomas is not included in the above data set because there are only 8 statements from him in the generated corpus
“Other speakers” are petitioners and respondents (not justices)
“stopwords” are high frequency words like the, to, also, etc.
“words”, “unique words” do not include “stopwords”

Linguistic diversity is a coarse measure of a varied vocabulary. The chart below displays the total number of unique words divided by the total number of words.

Nothing too interesting or surprising here. Justices may use less words in their sentences for a variety of reasons. This data isn’t normalized to factor out introductions or interruptions. Nevertheless the trend appears to be that Justice Kagan and Alito’s sentence lengths are longer than the others and about equal to the petitioners and respondents.

Long words in the oral transcripts

18 or 19 letters is the typical length for the longest words (that are in the dictionary) used by the various speakers. This includes non-sitting justices where the data was available.

• Justice Alito – (18) misrepresentation
• Justice Thomas – (16) unconstitutional
• Justice Kagan – (18) misrepresentations
• Justice Rehnquist – (18) telecommunications
• Justice Sotomayor – (18) unconstitutionally
• Justice Ginsberg – (18) misrepresentations / telecommunications / disproportionately
• Justice Scalia – (19) unconstitutionality
• Justice Breyer – (18) representativeness / telecommunications / disproportionately / unconstitutionally
• Other speakers – (19) counterintelligence / unconstitutionality / extraterritoriality

Sentiment Analysis

Sentiment analysis can yield interesting results for corpus data though in this case there is not very good training material. One of the standard data-sets used for this are movie reviews, widely available and with clear negative and positive denotations. For more information about sentiment analysis there is some good information here and in these two articles. Applying this to oral arguments? Well let’s leave it as just one way to look at this data..

Once the sentiment engine is trained, each statement registers as either positive or negative based on the words that match closest to the language in positive and negative movie reviews.

I’m sure the error of margin is high and without training data for similar text I would hesitate to draw any conclusions. One that I might make from this is that petitioner and respondents tend to use more positive language than the judges on the bench. If that is a valid hypothesis this data certainly seems to validate it.

Laugh lines in the oral transcripts

Note: there are venturesome academic papers like this one from the Communication Law Review that address laughter in the SCOTUS courtroom. I make no attempt to go into that depth here though my results more or less agree with previous studies on this topic.

It’s not uncommon to get laughter after a statement from the bench, this is denoated in the transcripts as either [laughter] or (laughter). This chart displays the total number of laughter lines by sitting Judge

As usual Thomas is excluded for generally not speaking when he sits on the bench.
Here is the same data but instead of the total number of [Laughter] lines it divides it by the number of statements for each justice.

Justice Breyer appears to be funnier by this measure.

Additional Notes

Unfortunately it’s impossible to cleanly extract the argument data from PDFs. Older transcripts have the Justice’s remarks labeled as “QUESTION”; without specific name references the data had to be discarded. Some transcripts have the Justice’s name spelled incorrectly, for example: JUST SCALIA or JUDGE SCALIA

Here is an example of where there is wrong attribution:

MR. LANDAU: JUSTICE O'CONNOR: Your Honor, that is not -And I think it is conceivable that the Florida court was correct that you could draw the line some way and say contracts that are void should be handled differently.

For this reason don’t take these results too seriously though hopefully the errors are down in the noise (I have no desire to go through and correct them).

The pdfs were parsed and the data was generated with two crufty python script, they are on my github if you want to look at this data yourself. If you make any improvements please let me know!

• Total number of male name login attempts: 17,804
• Unique male names: 1,565
• Logest male names: christopher, bartholomew, constantine

• Total number of female name login attempts: 22,483
• Unique female names: 2,217
• Logest female names: alexandrina, constantine

“billy” wins the prize; the corpus for “names” has it as both male and female. Also the corpus has “temp” as a male name which appears in the top 1% for obvious reasons.

What about all logins? For these we will just look at the top 0.1%.

• Total number of login attempts: 125,588
• Unique logins: 35,627
• Longest login name: fidelu142muiesteaua8642jet184

Conclusion: if you name your kid billy or test make sure he uses certificate authentication.

Construction

Once I had the circuit working on a breadboard it was just a matter of finding a prototype board that fit and some ugly soldering to glue it all together. In my case I had these parts lying around (including the tin) but if you wanted to buy everything it would cost between $5-$10.

• PacMan ghost candy tin
• Prototype Board
• A couple stand-offs and screws
• Two 1k potentiometers
• Piezzo Speaker
• Batter holder
• Coin cell battery
• DIP socket
• ATTINY85 Microcontroller(8k of flash, internal clock @ 8MHz)
• Push-button switch

Other materials..

• AVR SPI programmer or an Arduino to program the attiny
• A drill with a decent bit to cut through the tin
• Musescore sequencing software (free)
• Soldering iron, wires, a free afternoon, etc.

The circuit is only slightly more complicated than the musical greeting card. Two 1K potentiometers are used to mix the two square waves into one speaker. The push switch is connected to the external interrupt pin which is set low when pressed. On the prototype board the switch is wired on the opposite side of the circuit so that the speaker faces down when placed in the candy tin. This makes it louder by drilling holes in the back piece (see below). Standoffs are used to prop up the non-speaker side of the circuit.

Nothing is needed to hold the the circuit board in the tin since the button keeps it in place and there isn’t a lot of extra room when it is put together.

Software

(for more on the PlayTune library see my earlier post on using the PlayTune library with an Arduino)

Like the musical greeting card we will use the PlayTune library to play the melody and the xml2h.py to handle the musical conversion. The conversion takes a single track and converts it into two byte arrays of pitches and delays. The pitch values are a function of clock frequency and the prescaler.

The total size of the program ends up being around 2k so there is plenty of room to add more songs if you are inclined.

I used three midi files for the songs and loaded them into musescore. In the application it was necessary to transpose it an octave, other than that there wasn’t much else to do since these songs are already two-tracks which is exactly what we want for the attiny.

After saving the midi file as a MusicXML file a header file is created for each song. These header files are what the PlayTune library uses for tone and delay values.

The prescale values scale the frequency of the clock by a power of 2. Ideally you want the lowest value given in the list though TIMER0 only supports values of 1024, 256, 64 and 8 so for the first part “64″ is chosen.
For TIMER1 (part2) the lowest number can be selected to give the highest timer resolution. The reason this is important is because the frequency of the square waves generated on the two microcontroller pins are only an approximation of the note frequency. Higher frequency == greater timer resolution == better pitch accuracy.

The AVR is immediatly put into power-down sleep mode. When an external level change on the INT0 pin is detected (button press) the ISR routine will run which will play one of the three tunes.

#include <avr/io.h> #include <avr/pgmspace.h> #include <avr/interrupt.h> #include <avr/sleep.h> #include <util/delay.h> #include <avr/interrupt.h> #include "playtune.h" #include "songs/mspacman-acti-they-meet-attiny.h" #include "songs/mspacman-game-start-attiny.h" #include "songs/mspacman-actii-the-chase-attiny.h" int main(void) {// setup interrupt GIMSK |= (1<<INT0); // INT0 enabled for interrupts while(1) {set_sleep_mode(SLEEP_MODE_PWR_DOWN); sleep_mode(); }return(0); }volatile uint8_t tune = 0; ISR (INT0_vect) {PlayTune theymeet0(0,MSPACMAN_ACTI_THEY_MEET0); PlayTune theymeet1(1,MSPACMAN_ACTI_THEY_MEET1); PlayTune gamestart0(0,MSPACMAN_GAME_START0); PlayTune gamestart1(1,MSPACMAN_GAME_START1); PlayTune thechase0(0,MSPACMAN_ACTII_THE_CHASE0); PlayTune thechase1(1,MSPACMAN_ACTII_THE_CHASE1); switch(tune) {case 1: while ( theymeet0.isPlaying() || theymeet1.isPlaying() ) { theymeet0.playNote(); theymeet1.playNote(); _delay_ms(65); }break; case 2: while ( gamestart0.isPlaying() || gamestart1.isPlaying() ) { gamestart0.playNote(); gamestart1.playNote(); _delay_ms(15); }break; case 3: while ( thechase0.isPlaying() || thechase1.isPlaying() ) { thechase0.playNote(); thechase1.playNote(); _delay_ms(65); }break; }if (tune == 3) {tune = 1; } else {tune++; }}

Usually that number is closer to zero :)

Other applications? Maybe a “musical throwy”, It’s almost cheap enough :)

Here is “You’ve got a friend in me” played on an attiny85 and two speakers converted from a midi version of the song:

Materials

• Attiny85 – Thru-hole, no frills AVR running at 8MHz $1.62
• Piezzo speaker(s) – I had some of these lying around $1.62/ea
• Battery holder - Plastic, thru-hole.  $1.06
• Coin cell battery – You can probably find cheaper ones.  $0.50

Total: $4.80 (cheaper than Hallmark)

(I’m going to use two speakers for mine bringing the cost up to $6.42)

Other misc materials

• Programmer for the attiny85 (you can use an Arduino if you have one)
• Soldering iron, solder, wires, etc
• Greeting card?

Construction

There are only 6 connections to solder, 5 if you tie black speaker wires together.  I set all pins that are not being used to low to be used as ground connections.  I would avoid connecting anything to the pin right beneath your +3V connection to avoid shorts.

Using a poor-man’s paper switch to complete the connection for the battery the tune only plays once and stops (the AVR is put into power-down mode after the song finishes).  There is plenty more that could be done here including using more pins for LEDs, etc.

It would also be possible to mix the two outputs into one speaker with some resistors, this was easier though.

Creating the music

There are many places to find midi tracks, the challenge is finding one that will work for one or two parts.  With Musescore you can merge multiple parts together if they don’t overlap.  This example I’ll use “You’ve got a friend in me”, the Disney ToyStory theme-song.

First thing to do is to clean it up in musescore, copy the “zap chords” plugin into your Musescore plugin directory.

Loading the midi file in Musescore

After removing the chords, transposing it, increasing some note lengths in the second part and cleaning it up:

After running chord-zap and some transposing

Looks pretty, right? Now it needs to converted into something that the Attiny85 AVR chip can read.
To do that we will use the xml2h.py script to create a header file for the PlayTune library.

Unlike running this for the Arduino in the last post we select a clock frequency of 8MHz.

Selecting Prescaler Values

Prescaler values of 2,4,16,32,128,512 are ONLY valid for the attiny85 TIMER1

The attiny85 has two timers each with their own prescaler value.  The prescaler value is an integer clock modifier that will divide the clockspeed (in our case 8MHz) so the timer runs slower.  On one hand this allows us to play lower frequencies, on the other it does not give us as much resolution resulting in less precise frequencies for the tones.  In general you want to select the lowest pre-scaler value that is shown in the list that the timer supports.

For the attiny85 TIMER0 supports 1,8,64,256,1024.  TIMER1 supports 1,8,16,32,64,128,256,512,1024,2048,4096

For the arduino atmega chips you should only use prescale values of 1,8,64,256,1024.

Software

Here is the code that plays the song using the PlayTune library, you can see more details about it here.

#include <avr/io.h> #include <avr/pgmspace.h> #include <avr/interrupt.h> #include <avr/sleep.h> #include <util/delay.h> #include "playtune.h" #include "songs/gotafriend.h" int main(void) {// set everything low that we aren't using DDRB |= (1<<PB2) | (1<<PB4) | (1<<PB3) | (1<<PB5); PORTB &= ~(1<<PB2) | ~(1<<PB4) | ~(1<<PB3) | ~(1<<PB5); PlayTune p0(0,PART0); PlayTune p1(1,PART1); while ( p0.isPlaying() || p1.isPlaying() ) { p0.playNote(); p1.playNote(); _delay_ms(70); }set_sleep_mode(SLEEP_MODE_PWR_DOWN); sleep_mode(); return(0); }

After the song is finished the Attiny85 will be put to sleep in low-power mode.

There are already plenty of other ways to generating tones using an Arduino.   You can play music from a midi input there is a library for playing PCM audio and similar projects for playing music using all three Arduino timers simultaneously. I’ve also played around with AVR sound generation before in previous posts.

Note: if you just want to use the pre-generated header files in the songs/ directory you can download the lib and hook up your Arduino to a speaker to play tunes..

What you need to get started

• Arduino (or an atmega328, atmega168)
• A speaker and some potentiometers to mix the inputs.
• A midi file you want to convert
• Python and the beautifulsoup library for parsing xml
• Sequencing software like the Musescore (free)  that will also convert the midi file to musicxml
• “xml2h.py” which parses a musicxml file and generates a header file with notes and delays for the arduino
• PlayTune library that will play the song on the atmega for any tune for up to three tracks simultaneously.

The result..

How music is generated using an Arduino

On the Arduino and in general on the atmega168 and 328 there are 3 timers that operate independently of each other in hardware.   The timers count up (starting at 0) to either 255 or 65535 and reset to zero.  When these timers are put into “CTC” (clear on timer compare) a pin is toggled every time the counter matches a specific value called the “match value”. The speed of the timer depends on two things, one is the speed of the clock ( for the Arduino this is 16MHz ) and a clock prescaler. The “prescaler” is an integer that can be set that will slow down the timer by a factor of 1024, 256, 64, or 8. Once the timer is setup in CTC mode and a match value is specified there will be square wave generated on a pin at a frequency proportional to how frequently the timer hits the match value.

Here are the match values that correspond to different pre-scalers and their corresponding piano notes at 16MHz. (This is from a spreadsheet I made which comes in handy especially when dealing with different clock frequencies)

Let’s start making music..

Using the PlayTune library

Copy playtune.cpp and playtune.h to a “PlayTune” directory in the libraries folder of your Arduino install.  Once it’s copied there you should see it in the “import library” list under “Sketch”.

To initialize PlayTune pass into the constructor what timer you want to use, what prescaler, a list of match values and a list of corresponding delays.

• “isPlaying()” lets us know if there are still notes left to play for the part. When it’s finished it will reset back to the beginning.
• “playNote()” will iterate over the note list, playing the tone that corresponds to the match value or a rest if zero.

By default the PlayTune lib will play a scale from middle C to tenor C if you just pass in the timer you want to use.

In this example we will play the scale on all three timers simultaneously.
(connect your speaker to pins 6,9 and 11)

#include #include void setup(void) {PlayTune p0(0); PlayTune p1(1); PlayTune p2(2); while (p0.isPlaying() || p1.isPlaying() || p2.isPlaying() ) { p0.playNote(); p1.playNote(); p2.playNote(); _delay_ms(500); }} void loop(void) {}

Note that the _delay_ms(500) function requires the additional include of util/delay.h.

Do not use the Arduino “delay()” function! It will cause problems because it uses TIMER0 which is being used by the lib for generating the tone.

So that was fun (but boring).. Now on to creating some more interesting music…

Midi, musescore and xml2h.py

Musescore is a wonderful free sequencing tool. Using that you can open midi files, adjust them, and convert them to musicxml.

“xml2h.py” is a script I wrote that takes a musicxml file and generates an header file that tracks two arrays for up to 3 parts, one for note values and another for delays (rests are note values equal to zero)

The conversion from midi to arduino tones is a bit of a hack :/
xml2h.py will do its best to:

• pick the best prescaler
• ignore chords
• combine voices if it can
• handle tied notes

I’ll walk through a video game theme as an example (it works well since the songs are simple). There are a number of sites out there with video game midi files most of them look like they were designed in 1997. Below I’ll use the Super Mario Brothers theme. In this score there are 4 parts, we will convert the first three to play on the Arduino.

SMB theme in Musescore

There are some limitations to keep in mind for the midi conversion:

• If notes span multiple clefs then you there will probably be too large a range for the resolution of the timer, not many (simple) songs have this problem though.
• When you convert you will need to select between 1-3 parts to be played simultaneously.
• Chords will be ignored by the xml2h.py script.
• If there are more than voice on a part the script will try to merge them together.

I wrote a plugin called “chord zapper” for musescore that will (attempt to) remove all the chords. Copy this to the musescore plugin folder, select all, run the plugin to get rid of all the chords if you want to get a better sense of how the song will sound when converted. Whether or not you remove chords the “xml2h.py” script will ignore them, taking the top note in the chord if there are any.

zapping all the chords (optional)

When you are satisfied save the midi file save it as a “MusicXML” file and run it through xml2h.py.
The script takes the xml plus the name of the header file you want to write out.

For example:

Pick the clock frequency. This defaults to 16MHz which is what you want for the Arduino
Pick the part you wish to process (part0 corresponds to the top clef in the midi file)
Pick the prescaler, the script will give you a subset of prescalers, you should select the lowest one that is supported. (see note on prescalers below)

If it finishes without error you will end up with a header file that contains some arrays and macros for passing them into the PlayTune constructor.

Here is what the resulting header looks like, the macros for PART1, PART2, PART3 are used in the sketch below:

#include #include #include void setup(void) {PlayTune p0(0,PART0); PlayTune p1(1,PART1); PlayTune p2(2,PART2); while (p0.isPlaying() || p1.isPlaying() || p2.isPlaying() ) { p0.playNote(); p1.playNote(); p2.playNote(); _delay_ms(50); }} void loop(void) {}

Assuming you are converting two parts and the first part will be using Timer0 and the second part Timer1 pick the lowest value from the two sets that show up in the prescaler selection.

The only difference between this sketch and the last one is the additional include for the “smb.h” header file, the addition of the PART# macros used in the PlayTune constructors and a smaller delay.

The value of the delay will determine the tempo of the song and corresponds to the delay between each tick in the MusicXML file or the shortest note.  Usually this is either 1/32 or 1/64.

If you have questions about the lib or if you create more songs let me knowand I will add them to collection.

Selecting Prescalers

In general you want to select a low value for a prescaler to give you the highest resolution (to match the pitch of the song as close as possible).
The two timers on the attiny support different prescalers though so you need to be aware of what timer you are using (0 or 1) and what prescalers it supports.

Timer0 supports:  1024, 256, 64, 8
Timer1 supports: 1024,512,256,128,64,32,16,8

There are a few other guides out there already for this, I’ll just add this one to the pile. This assumes you want to program an AVR microcontroller with gcc and use something like the USBtiny or another programmer that AVRDude supports.

Everything you need will be installed in your home directory.

I’ll be building this on a debian box so I’ll assume you have apt to prime your build environment. No reason why the same can’t be done using another distro.

Installing the avr toolchain

mkdir ~/src # for everything sudo apt-get install build-essential texinfo # for avrdude sudo apt-get install flex byacc libusb-dev # for gdb sudo apt-get install libncurses5-dev # for simulavrxx sudo apt-get install tcl-dev cvs autoconf \libtool swig g++ zlib1g-dev texlive-base

Setup search paths

export PATH=$HOME/bin:$PATH export C_INCLUDE_PATH=$HOME/include export LIBRARY_PATH=$HOME/lib export LD_LIBRARY_PATH=$HOME/lib

IMPORTANT: If you have other versions of gmp and mpfr installed and you don’t set these variables you will certainly run into issues.

Compile and install binutils

Download the latest version of bintools:
http://ftp.gnu.org/gnu/binutils/?C=M;O=D

cd ~/src wget http://ftp.gnu.org/gnu/binutils/binutils-2.19.1a.tar.bz2 tar jxvf binutils-2.19.1a.tar.bz2 cd binutils-2.19.1/ ./configure --prefix=$HOME --target=avr \--enable-install-libbfd --disable-werror make && make install

Compile and install gmp/mpfr/mpc

Download the latest gmp/mpfr/mpc from

ftp://gcc.gnu.org/pub/gcc/infrastructure/

cd ~/src wget ftp://gcc.gnu.org/pub/gcc/infrastructure/gmp-4.3.2.tar.bz2 tar jxvf gmp-4.3.2.tar.bz2 cd gmp-4.3.2/ ./configure --prefix=$HOME make && make install cd ~/src wget ftp://gcc.gnu.org/pub/gcc/infrastructure/mpfr-2.4.2.tar.bz2 tar jxvf mpfr-2.4.2.tar.bz2 ./configure --prefix=$HOME make && make install cd ~/src wget ftp://gcc.gnu.org/pub/gcc/infrastructure/mpc-0.8.1.tar.gz tar zxvf mpc-0.8.1.tar.gz cd mpc-0.8.1 ./configure --prefix=$HOME make && make install

Compile and install gcc

Grab the latest GCC source from
ftp://ftp.gnu.org/gnu/gcc

wget ftp://ftp.gnu.org/gnu/gcc/gcc-4.3.6/gcc-4.3.6.tar.gz tar zxvf gcc-core-4.3.6.tar.bz2 mkdir gcc-4.3.6/build cd gcc-4.3.6/build ../configure --target=avr --prefix=$HOME \--disable-nsl --enable-languages=c,c++ \--disable-libssp make && make install

Compile and install avr-libc

Grab the latest avr-libc from

http://download.savannah.gnu.org/releases/avr-libc/

I downloaded the latest out of CVS since there was a bug in 1.7.1.

cd ~/src wget \http://download.savannah.gnu.org/releases/avr-libc/avr-libc-1.7.2rc2252.tar.bz2 tar jxvf avr-libc-1.7.2rc2252.tar.bz2 mkdir avr-libc-1.7.2rc2252/build cd avr-libc-1.7.2rc2252/build export CC=avr-gcc ../configure --build=`../config.guess` --host=avr --prefix=$HOME make && make install

Compile and install avrdude

Grab the latest avrdude from

http://nongnu.askapache.com/avrdude/

cd ~/src wget http://nongnu.askapache.com/avrdude/avrdude-5.10.tar.gz tar zxvf avrdude-5.10.tar.gz cd avrdude-5.10 export CC=gcc ./configure --prefix=$HOME make && make install

Compile and install gdb

Grab the latest gdb from

http://ftp.gnu.org/gnu/gdb/

cd ~/src wget http://ftp.gnu.org/gnu/gdb/gdb-7.3.1.tar.bz2 tar jxvf gdb-7.3.1.tar.bz2 cd gdb-7.3.1/ export CC=gcc ./configure --target=avr --prefix=$HOME \--disable-werror make && make install

Compile and install SimulAVR

Grab the latest SimulAVR from

http://savannah.nongnu.org/projects/simulavr/

cd ~/src git clone git://git.savannah.nongnu.org/simulavr.git cd simulavr ./bootstrap ./configure --prefix=$HOME --with-bfd=$HOME make && make install

Makefile template

Now that all of the tools are installed you will need to put together a makefile. The path of least resistance is to use a template, modify it to suite your needs and go. Happy Coding!

All of the custom javascript for the site is contained in a single js file that is loaded as a Django template. The reason for loading it as a template rather than as a static file is so that we can reference the STATIC_PREFIX location for static files as well as some other template goodies like dynamically referencing URLs. If we ever change where our static files are stored or decide on a different URL naming scheme they will be updated in the javascript dynamically.

JQuery Introduction

JQuery is a javascript library that that helps with some of the cross-browser compatibility issues that are common when writing javascript. It also handles the visual effects on the page and provides AJAX request methods for dynamic updates.

The js file for the JQuery and JQuery-UI libraries need to be included in the base template after it is copied to the static file directory. The UI library is an extension of the base JQuery library and is used specifically for the “puff” animation that happens when you select an answer.

wordconfuse/templates/base.html:

<script src="{{ STATIC_PREFIX }}js/jquery-1.5.1.min.js" type="text/javascript" charset="utf-8"></script> <script src="{{ STATIC_PREFIX }}js/jquery-ui-1.8.9.custom.min.js" type="text/javascript" charset="utf-8"></script>

There are a number of JQuery functions that are used in the app:

• jQuery.each() – used to iterate through a collection of objects
• jQuery.ajax – for all AJAX posts and requests made to the server
• fadeout / fadein / show / hide – used for all the fade-in and fade-out animation and to hide and show the various game screens
• puff – UI effect when a definition is clicked
• click – setup click handlers for buttons and links
• html – dynamically update html on the page
• animate – used to move the correct definition after selection

Some javascript built-ins are used to handle the delays and timer for the game:

• setTimout – Used to add delays between events on the screen
• setInterval – Called once per turn to update the countdown timer and so we know when the player has run out of time.
• clearInterval – Stops the counter setup by setInterval and is called multiple times both when the player runs out of time and when when he clicks one of the answers.

Some general javascript/JQuery tips

If you are coming to javascript/jquery from other languages there are a couple things that are essential to before jumping in.

• javascript uses function level scoping, not block level scoping

  How many derps does this (awful) code print to the console?

  for (var i=0; i<10; i++) {for (var i=0; i<10; i++) {console.log(i + "derp"); }}

  If you guessed “10 derps”, you are correct. The variable “i” in both loops share the same scope. Don’t re-use variable (object) names inside a function. Objects in a function share the same namespace.

• JQuery calls are asynchronous most of the time

  It’s a common pattern in JQuery to fade something out, update content, and fade it back in.
  Say the content we want to fade out and fade in is between the following div tag:

  <div class="foo"> ... </div>

  An intuitive procedural approach to doing this might be:

  $(".foo").fadeOut('slow'); $(".foo").fadeIn('slow'); $(".foo").html('herp derp');

  Calls are asynchronous so our “foo” div will be fading out, fading in, and herp derp’ing all at the same time! That’s probably not what you want.

  Instead, you should do this:

  $(".foo").fadeOut('slow', function() {$(".foo").fadeIn('slow', function() {$(".foo").html('herp derp'); }}

  By using these closures as callbacks to the fade functions we can be sure that everything is executed in the proper order.

Another common pattern is to display some text, wait X number of seconds and then do something else. There isn’t a sleep() function in javascript otherwise we would be blocking the execution such that the browser would stop responding to updates. Use the setTimeout() function which takes a function reference as a callback. You will see this in the WordConfusion javascript code whenever a delay is needed in the game.

$(".foo").html("herp"); setTimeout(derp, 500); function derp() {$(".foo").html("derp"); }

The code for WordConfusion

Page loading, the spash screen

The function passed in to the jquery $(document).ready() function will execute code when the browser makes a new request to document root.

var gameover = { ... }var high_scores = { ... }var game = { ... }$(document).ready(function() {if (typeof console == "undefined") {console = { log: function() {} }; }// on the main page load we show the splash screen // and register start and highscores for clicks $('.new').click( function() {$('.splash').fadeOut( '1000', function() {game.startGame(); }); }); $('.highscores').click( function() {$('.splash').fadeOut( '1000', function() {high_scores.showscores(); }); }); $('.splash').fadeIn('500'); });

The first thing that’s being done on the load is to make sure console logging is off if there isn’t a console connected. Next we set up two click handlers for highscores and starting a new game. Finally, the last line in the fades in the spash screen giving the user the option of starting a new game or viewing the highscores. At this point code execution stops and waits for the user to click one of the buttons.

Above the code $(document).ready() are the objects that hold the code for the game logic, these objects for gameover, high_scores and game and are the only ones in the global namespace. The rest of this tutorial will talk about each of these objects and the code in them for handling the screens and game logic.

Displaying the highscore screen

Displaying the highscore table will be the simplest code path so it’s probably a good place to start. When the user clicks the high_score button the showscores() function will run.

var high_scores = { showscores: function() {$.ajax({ url: '{% url hs %}', cache: 'false', dataType: 'html', async: 'false', success: function(data) {$('.hs').show(); $('.hs-content').html(data); $('.playagain').click(function() {$('.hs').fadeOut('slow', function() {game.startGame(); }); }); }, error: function() {console.log("ERROR: hs"); }, }); }}

First notable thing in this javascript code is some Django syntax. The {% url hs %} is Django template syntax for the URL corresponding to the highscore view. If the URL corresponding to that view ever changes the javascript will be updated dynamically.

The showscores() function executes a jquery ajax call which sends a get request from the browser (without re-loading the full page). The results of this request are put in the data variable which is displayed between the div tags that have the “hs-content” class by $(‘.hs-content’).html(data).
Remember the div tags with the “hs-content” class is in the index.html template described in part 3 of the tutorial.

<div class="hs-content"> ... </div> ^ the html returned by the "hs" view will be inserted here

Displaying the main game screen

Setting up the game

var game = { ... ROUNDS: 10, ANSWERS: 4, TURNTIME: 9, startGame: function() {this.s=null; //setInterval (for the countdown timer) this.turn=1; // turn count this.sec_left=null; // number of seconds left for the countdown timer this.wd= null; // current word group in quiz this.count=0; // number of correct answers this.submitted = true; // setup click handlers for answers $('.ab').each( function() {$(this).click( game.answer_click ); }); // grab the word list $.ajax({ url: '{% url get_words %}', cache: 'false', dataType: 'json', async: 'false', success: function(data) {game.wl = data; // preload images jQuery.each(game.wl, function(index,value) {var objImage = new Image(); objImage.src ='{{STATIC_PREFIX}}img/words/'+value['q']+'.png'; if (index == game.wl.length - 1) {// last image has loaded game.new_turn(); }}); }, error: function() {console.log("ERROR: get_words"); }, }); }, ...

The startgame function sets up the game. When a new game starts some variables that are local to the game object are created. The names should be self explanatory but the following deserve some additional explanation:

• sec_left – Within the game object this will keep track of the total number of seconds left per turn, currently this is configured to count 9 seconds.
• wd – When a request is made to get a new list of words by hitting the get_words URL this variable will contain the current word information as the game iterates through the list.
• count – This will track the total number of correct answers which is posted to the server when the game is over.
• submitted – This variable that is global to the game object will be set to true as soon as an answer is selected or when there is no more time left for the current turn. This is so that the code can make sure the user doesn’t submit an answer twice. After an answer is clicked or time has run out it will be set back to False when the next word and definitions are shown

After some variables that are global to the game object are initialized, click handlers are setup for the answers.

$('.ab').each( function() {$(this).click( game.answer_click ); });

For every div block in the index.html template that has the “.ab” class, a click will execute code referenced by game.answer_click.

The last setup task is to fetch the word data and put it into the wl variable which is global to the game object. Once the data is fetched the code iterates through the wl object and caches all the images so they are displayed in a timely manner as the game runs. When the last image is cached the code in game.newturn is called to begin the game.

Starting the Game

new_turn: function() {$('.nt').hide(); clearInterval(game.s); if (game.turn > game.ROUNDS) {console.log('game over'); $('#a0').unbind(); $('#a1').unbind(); $('#a2').unbind(); $('#a3').unbind(); game_over.showscreen(); } else {$('.game-questions').fadeOut( 'slow', function() {$('.game-info').html('Turn ' + game.turn + ' / 10'); game.wd = game.wl[game.turn-1]; time = game.unix_time(); game.sec_left = game.TURNTIME; $('.ct').html(game.sec_left); game.s = setInterval( function() {if (game.sec_left <= 1) {if (game.submitted) {return false; } else {game.submitted = true; }clearInterval(game.s); // stop the countdown timer $('.nt').css({ 'backgroundImage': 'url("{{STATIC_PREFIX}}img/outoftime.png")', }); $('.nt').fadeIn('slow',function() {game.show_answer(); }); } else {game.sec_left = game.TURNTIME - (game.unix_time() - time); if (game.sec_left >= 1) {// just to be paranoid.. $('.ct').html(game.sec_left); }} }, 1000); $('.qb').html('<img src="{{STATIC_PREFIX}}img/words/' + game.wd['q'] + '.png" />'); // reset the answer board $('.ab').each( function() {$(this).removeAttr("style"); }); for (var j=0; j<game.ANSWERS; j++) {$('div#a' + j + ' .answer').html('<p>' + game.wd['a'][j] + '</p>'); }$('.game-questions').fadeIn( 'slow', function() {$('.game').fadeIn('500'); // for the first time game.submitted = false; // finally we allow clicks }); game.turn++; }); }}, unix_time: function() {return Math.floor(new Date().getTime() / 1000); }, }

The new_turn function is called 10 times, once for every turn. If there are still rounds left in the game it will update the board with the definition, answers and set up a timer for the turn. If there are no rounds left it will end the game and call the show_screen function in the game_over object.

When a new game is started a few things are done:

• The game.wd = game.wl[game.turn-1]; takes the list of word data that was fetched earlier from the URL get_words and puts it in the wd object which will be used for the current turn.
• game.s = setInterval( function() {…},1000 ) will execute code every second to update the countdown timer and check to see if there is any time left. One important thing to note in this code is that it uses the system clock, and doesn’t rely on setInterval for the time. This is a more accurate way to track of time on the browser since if the browser is busying doing other things the function may not run precisely every wall-clock second
• After the interval is set the board needs to be updated with the words and definitions for the current turn data in the wd object. Iterating through the question boxes for the words and the answer boxes for the definitions the appropriate images and text are faded in so that the user can select an answer.

Clicking on an answer

answer_click: function () {if (game.submitted) {return false; } else {game.submitted = true; }// check to see if the word is correct id = $(this).attr('id'); if (id == 'a' + game.wd['s']) {clearInterval(game.s); // stop the countdown timer game.count++; // increase the correct answer count $(this).css({ 'opacity' : '.6', 'background' : '#3366cc', }); $('.nt').css({ 'backgroundImage': 'url("{{STATIC_PREFIX}}img/goodjob.png")', }); $('.nt').fadeIn('slow',function() {setTimeout(game.show_answer, 300); }); } else {$('.nt').css({ 'backgroundImage': 'url("{{STATIC_PREFIX}}img/wronganswer.png")', }); $('.nt').fadeIn('slow',function() {setTimeout(game.show_answer, 300); }); }},

This is the code that is executed (via the click handler that was setup in startGame) when a player clicks on a defintion. The first if-else in this function is an important check to make sure that an answer wasn’t already clicked. If it was then we just return from it without doing anything.

The rest of this function checks the answer that was clicked against the answer that was read in from the get_words data. If it matches a “good job” is displayed on the screen and the show_answer function is called after about 300ms. If it doesn’t match a “wrong answer” is displayed and the show_answer function is also called. If the answer is correct the variable count (global to the game object) is incremented by 1. This keeps track of the number of correct answers.

Ending the turn, showing the correct answer

show_answer: function () {clearInterval(game.s); // stop the countdown timer // reset opacity $('.ab').each( function() {$(this).removeAttr("style"); }); var wrong_answers = []; for (var cnt=0; cnt < game.ANSWERS; cnt++) {if (cnt != game.wd['s']) {wrong_answers.push(cnt); }} jQuery.each(wrong_answers, function(index,value) {$('#a' + value).hide("puff",{},'1000', function() {if (index==wrong_answers.length-1) {//last wrong answer has faded out, //now move the correct one var pos0 = $('#a' + game.wd['s']).position(); // change the style $('#a' + game.wd['s']).css({ position: 'absolute', top: pos0.top, }).animate( {top: 160, left: 135, }, 500,function() {setTimeout(game.new_turn,900); }); }}); }); },

At end of every turn this function is called to clear out the wrong answers (whether or not the player clicked the correct one) and display the correct one. After the wrong answers are cleared using the JQuery UI “puff” effect and the correct answer is moved to the center using the JQuery animate effect. The new_turn function will be called again after 900ms to setup the board for the next word and definitions.

GameOver

There are two functions for the game_over object, one for posting the data to the game_over url and another for setting up the click handlers for when a new highscore is created. Starting with the showscreen function:

var game_over = { ... showscreen: function() {$('.game').fadeOut('slow', function() {$.ajax({ url: "{% url gameover %}", data: {'count': game.count }, type: 'post', async: 'false', success: function(data) {$('.gameover').show(); $('.gameover').html(data); game_over.button_events(); }, error: function () {console.log("ERROR: gameover"); }, }); }); }, ...

After the main game board fades out using the JQuery fadeout function an AJAX request is sent to the gameover URL containing post data for the number of words that were answered correctly. The request to the gameover URL will return html giving the player his score, the current highscores and a option to submit his name if he rates the high score table.
Once the html is displayed between the div tags in index.html with the “gameover” class, button_events() is called to set upt the click handlers for playing a new game and submitting a highscore.

button_events: function() {$('.playagain').click(function() {console.log("restarting game"); $('.gameover').fadeOut('slow', function() {game.startGame(); }); }); $('.hs_submit').click(function() {console.log("hs_submit clicked"); // simple length check before they // submit var username = $('#id_username').val(); if (username.length > 20 || username.length < 3) {console.log("invalid username: " + username.length); $('.error').html( 'Please use a username between 3 and 10 characters'); } else {console.log("submitting newhs") $('.error').html(); $.ajax({ url: "{% url new_hs %}", data: {'username':username}, type: 'post', async: 'false', success: function(data) {console.log("fading out"); $('.newhs').fadeOut('slow', function() {$('.hs-content').html(data); }); }, error: function() {}, }); }}); }

If the player clicks the link with the “playagain” class he will start over with a new game.

The other handler is for the submit button for a new high score with the “hs_submit” class. This will only show up if the score is high enough to be on the highscore table and is determined by the server when it sends the html back using the gameover template.

There is a small amount of client side javascript checking here to make sure the user submits a name between 3 and 20 characters.

if (username.length > 20 || username.length < 3) {console.log("invalid username: " + username.length); $('.error').html( 'Please use a username between 3 and 10 characters'); }

The client-side check is just a convenience for a coarse validation. It is never safe to rely on javascript validation since it can easily be bypassed by the browser.
After the username is submitted an updated highscore screen is returned from the “new_hs” URL that will be updated with the player’s name.

Summary

That covers all of the javascript for the screens and game logic for WordConfusion. I hope you found this tutorial valuable if you have any questions, suggestions, or find errors please feel free to send me mail or write in the comments below. Good luck!