Hardware Architecture
Intel Edison Kit for Arduino
Intel Edison is utilized as the only microprocessor unit in this project. It is a high performance, dual-core CPU and single core micro-controller supports complex data collection in a low power package. With integrated Wi-fi and USB port, this is the perfect prototyping board for our particular project.
USB Microphone
In the Intel Edison board, there are two ways to record audio, one way is recording through USB port, and another way is measuring through analog pin. When you plug a microphone breakout via the analog pin, the connection is analog audio. This means any wiring or connection problems can cause noise. Also, the sampling rate is very hard to control. However, the USB microphone is connected digitally. The signal has very unified sample rate and much less noise.
Intel Edison is utilized as the only microprocessor unit in this project. It is a high performance, dual-core CPU and single core micro-controller supports complex data collection in a low power package. With integrated Wi-fi and USB port, this is the perfect prototyping board for our particular project.
USB Microphone
In the Intel Edison board, there are two ways to record audio, one way is recording through USB port, and another way is measuring through analog pin. When you plug a microphone breakout via the analog pin, the connection is analog audio. This means any wiring or connection problems can cause noise. Also, the sampling rate is very hard to control. However, the USB microphone is connected digitally. The signal has very unified sample rate and much less noise.
Overall Architecture
Software Architecture
Project Directory Structure
|-/ubisound
|-----/data
|----------knock1.wav
|----------- . . .
|----------knock100.wav
|----------- . . .
|----------closedoor1.wav
|----------- . . .
|----------closedoor100.wav
|-----record.py
|-----feature_extraction.py
|-----create_datasource.py
|-----create_ml.py
|-----create_evaluation.py
|-----run.py
|-----data.csv
|-----/data
|----------knock1.wav
|----------- . . .
|----------knock100.wav
|----------- . . .
|----------closedoor1.wav
|----------- . . .
|----------closedoor100.wav
|-----record.py
|-----feature_extraction.py
|-----create_datasource.py
|-----create_ml.py
|-----create_evaluation.py
|-----run.py
|-----data.csv
Data Collecting - record.py
This is the code to record a 5 seconds wave file for data analytic. In this script, 'subprocess.call' is used to execute a command-line soundfile recorder, named arecord. This recorder is particular used for ALSA sound card driver.
Example: Arecord command to record 5 seconds '.wav' file with mono channel, 48000 Hz sampling rate, 16 bit little endian data
arecord -d 5 -f dat -c 1 -D hw:2,0, test.wav
In the python file, these commands are passed to subprocess by args. The script will wait for command to complete, then execute next line.
To record a wave file using this script, simply type:
Python record.py test
'test' is the file name to store the wave file. The file will be stored in './data/' director.
In our case, we named knock door as 'knock#.wav', close door as 'closedoor#' and noise as 'nothing#.wav'. 100 samples are record for each of these activities.
Feature Extraction - feature_extraction.py
This is the script to extract the features of the wave files, and store them in a single .csv file. Four features are implemented:
- Zero-crossing Rate - the rate of sign-changes of the signal during the duration of wave.
- Energy - the sum of squares of the signal values.
- Frequency-weighted sum of frequency between 400~500 Hz - the sum of frequencies in a particular frequency range.
- Frequency-weighted sum of frequency between 700-900 Hz - the sum of frequencies in another frequency range.
The script reads wave files from the data directory, extracts the features and stores them in a 'data.csv' file.
Then, the csv file needs to be uploaded to AWS s3 for creating datasource and machine learning.
Create Datasource - create_datasource.py
This script creates AWS datasource from AWS S3. Two datasource are created from the data sets. One for building machine learning model (80% of data), one for model evaluation (20% of data).
Build Machine Learning Model - create_ml.py
This script builds AWS machine learning model from the datasource. 'DataSourceId' needs to be replaced with the data source id created in the last step. The machine learning model is named as 'ubisound_ml'.
Real Time Prediction - run.py
This is the main application code. It houses all feature extraction methods, general application functions and wave file recorder.
Pseudo logic:
while(true):
-- record data.wav.
-- read audio.wav
-- audio_fft = fft(audio.wav)
-- energy = getEnergy(audio_fft)
-- if (energy > MIN_ENERGY):
------ZCR = getZCR(audio.wav)
------BAND1 = getBAND1(audio_fft)
------BAND2 = getBAND2(audio_fft)
------result = PREDICT(ZCR, BAND1, BAND2, ENERGY)
------sendMessage(result)