Image

SEAQUENCE

Final Technological Solution:

Due to confusion from the children during our prototype testing with differentiating responses from sequences and success from failure, we decided to change our feedback mechanisms. This meant making a clearer differentiation between stimuli for the showing of sequences and the feedback for responses as well as adding standard messages to indicate success and failure. To make this as clear as possible, our final project technological solution combines robotics with pseudo-robotics. Specifically, the built-in-map, sounds, showing of sequences, and success-failure feedback were all faux-robotic (a PowerPoint presentation and keyboard sounds over a Bluetooth speaker) operated by one of our team members during testing.

Hardware wise, our project has three separate colored stations, or EV3 bricks. Each station has a button and a flag of the proper color fixed to the motor. Each station is capable of rotating the flag to indicate that the button has been press by the children and sending information wirelessly. These four stations are all in turn controlled by a Raspberry Pi that acts as the “Simon”. This central computer runs the code that receives information from the four stations to piece together a collective response. Then, it analyzes the response to determine success or failure, which it then indicates on the green and red LEDs attached. Additionally, the Pi has a pushbutton attached that acts primarily as a kill-switch to start a level over.

Software wise, the code is parceled into four files, one for each level, run by the Raspberry Pi. For each file, the Pi first initializes all variables and devices for the game before defining two sub-functions. The first is for recording responses when only one color input is expected. The second is a function for recording two colors at once when the game requires the children to press two buttons simultaneously. Both functions respond by raising and then lowering the appropriate flag. After initializing all devices and functions, the code awaits the kill-switch before continuing. From there, each level, with the exception of the first, is essentially the same.

For the first level, the following loop contains an if statement waiting for the kill-switch to be pressed. This would end the level. Otherwise, the code runs the receive functions. This level is used strictly to get the children comfortable using the buttons and interacting with the robot. It can be concluded at any point.

The remaining levels contain a more complicated structure of embedded loops. The highest level loop iterates over the ten sequences that make up each level. Within this, there is a small section of code that instructs both LEDs to blink a number of times equal to the sequence number. This is show the operator how far the code has progressed in case the print command stops working. Within each iteration of the sequence loops, an embedded while loop then contains the rest of the logic. First, the code utilizes three if statements that determine whether the response is correct or incorrect. When a correct solution triggers the first if statement, the loop will be broken and the higher level loop will iterate to the next sequence. When an incorrect response of equal length to the solution is found or the kill switch is pressed triggering the second or third condition, respectively, the loop resets the response variable and begins again from scratch. If none of these criteria are met, the code continues waiting for sensory input using one, or both, of the subprograms defined above.

Testing:

Before the test, we ran all the levels and everything work pretty well. However, when it comes to the kids, the system went wrong. Reaction of the buttons became very slow and since the kids often blocked the motion of motors with their hands and bodies, motors often couldn't get back to the initial position. We tried to ran the program again, reboot the IDE, but all didn't help a lot. At last, we restarted all of the LEGO kits. It took a while and our teammates tried to play other games with the kids. After the restart, everything went much better. One reason we thought that  might cause the problem was that since kids often pressed each button for many times before the motor response, the program might stuck because it received to many commands in a short time. Also, it might be a network issue for the IP address of our LEGOs changed several times and this never happened before. We learned a lot from the testing, even though we made many updates after the first test, we are still not well-considered enough on the kids' perspective. Besides, we should have a back up plan so that when something goes wrong, we could also let the kids to try some parts of our games.

Pictures:

Image Image

Code:

(there's a lot but it's pretty repetitive)

Common for all levels:

import RPi.GPIO as GPIO

def main():
    # define hardware
    gpg = Device("this")
    killSwitch = gpg.init_button_sensor("AD2")
    Correct_LED = 17
    GPIO.setmode(GPIO.BCM)
    GPIO.setup(Correct_LED,GPIO.OUT)
    GPIO.output(Correct_LED,GPIO.LOW)
    Wrong_LED = 27
    GPIO.setmode(GPIO.BCM)
    GPIO.setup(Wrong_LED,GPIO.OUT)
    GPIO.output(Wrong_LED,GPIO.LOW)
    print('Pi Initiated')
    stationR = Device('172.16.216.92')
    motorR = stationR.LargeMotor('outA')
    buttonR = stationR.TouchSensor('in1')
    print('Red Initiated')
    stationY = Device('172.16.143.207')
    motorY = stationY.LargeMotor('outA')
    buttonY = stationY.TouchSensor('in1')
    print('Yellow Initiated')
    stationG = Device('130.64.142.80')
    motorG = stationG.LargeMotor('outB')
    buttonG = stationG.TouchSensor('in1')
    print('Green Initiated')
    tag = 0

    # "receive" function
    # 1 = red
    # 2 = yellow
    # 3 = green
    def receive( response, i ):
        tag = 1
        if buttonR.is_pressed == 1:
            motorR.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
            sleep(.6)
            motorR.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
            response.append(1)
            print(response)
            i=i+1
            sleep(1)
        elif buttonY.is_pressed == 1:
            motorY.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
            sleep(.6)
            motorY.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
            response.append(2)
            print(response)
            i=i+1
            sleep(1)
        elif buttonG.is_pressed == 1:
            motorG.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
            sleep(.6)
            motorG.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
            response.append(3)
            print(response)
            i=i+1
            sleep(1)
        return

    # "receive2" function
    # 4 = red and yellow
    # 5 = red and green
    # 6 = yellow and green
    def receive2( response, i, tag ):
        if buttonR.is_pressed == 1 and buttonY.is_pressed == 1:
            motorR.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
            motorY.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
            sleep(.6)
            motorR.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
            motorY.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
            response.append(4)
            print(response)
            sleep(1)
            tag = 1
        elif buttonR.is_pressed == 1 and buttonG.is_pressed == 1:
            motorR.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
            motorG.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
            sleep(.6)
            motorR.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
            motorG.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
            response.append(5)
            print(response)
            sleep(1)
            tag = 1
        elif buttonY.is_pressed == 1 and buttonG.is_pressed == 1:
            motorY.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
            motorG.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
            sleep(.6)
            motorY.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
            motorG.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
            response.append(6)
            print(response)
            sleep(1)
            tag = 1
        return

    # wait for go
    GPIO.output(Correct_LED,GPIO.HIGH)
    print('Waiting for go...')
    while True:
        if killSwitch.is_button_pressed():
            GPIO.output(Wrong_LED,GPIO.HIGH)
            sleep(.5)
            GPIO.output(Correct_LED,GPIO.LOW)
            GPIO.output(Wrong_LED,GPIO.LOW)
            break

Level 1:

    # Level 1
    print('LEVEL 1')
    response = []
    i = 0
    while True:
        if killSwitch.is_button_pressed():
            print('Kill switch')
            GPIO.output(Correct_LED,GPIO.HIGH)
            GPIO.output(Wrong_LED,GPIO.HIGH)
            sleep(.5)
            GPIO.output(Correct_LED,GPIO.LOW)
            GPIO.output(Wrong_LED,GPIO.LOW)
            break
        else:
            receive2( response , i, tag )
            if tag == 0:
                receive( response , i )
                sleep(.2)
            tag = 0
    print('LEVEL 1 DONE')

if __name__ == '__main__':
    main()

Level 2:

    # LEVEL 2
    print('LEVEL 2')
    j = 1
    while j < 11:
        solutionA = [2]
        solutionB = [3]
        solutionC = [1]
        solutionD = [3]
        solutionE = [2]
        solutionF = [1]
        solutionG = [2]
        solutionH = [1]
        solutionI = [3]
        solutionJ = [2]
        i = 0
        response=[]
        print(j)
        k = 0
        sleep(.5)
        while k < j:
            GPIO.output(Correct_LED,GPIO.HIGH)
            GPIO.output(Wrong_LED,GPIO.HIGH)
            sleep(.1)
            GPIO.output(Correct_LED,GPIO.LOW)
            GPIO.output(Wrong_LED,GPIO.LOW)
            sleep(.1)
            k = k + 1
        while True:
            if (response==solutionA and j==1) or (response==solutionB and j==2) or (response==solutionC and j==3) or (response==solutionD and j==4) or (response==solutionE and j==5) or (response==solutionF and j==6) or (response==solutionG and j==7) or (response==solutionH and j==8) or (response==solutionI and j==9) or (response==solutionJ and j==10):
                #print(response)
                print('correct')
                GPIO.output(Correct_LED,GPIO.HIGH)
                sleep(.5)
                GPIO.output(Correct_LED,GPIO.LOW)
                j = j + 1
                break
            elif len(response)==1:
                #print(response)
                print('wrong')
                GPIO.output(Wrong_LED,GPIO.HIGH)
                sleep(.5)
                GPIO.output(Wrong_LED,GPIO.LOW)
                motorG.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                motorY.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                motorR.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                sleep(.5)
                motorG.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                motorY.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                motorR.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                response=[]
            elif killSwitch.is_button_pressed():
                print('Kill switch')
                GPIO.output(Correct_LED,GPIO.HIGH)
                GPIO.output(Wrong_LED,GPIO.HIGH)
                sleep(.5)
                GPIO.output(Correct_LED,GPIO.LOW)
                GPIO.output(Wrong_LED,GPIO.LOW)
                motorG.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                motorY.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                motorR.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                sleep(.5)
                motorG.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                motorY.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                motorR.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                response = []
            else:
                receive( response , i )
    print('LEVEL 2 DONE')

if __name__ == '__main__':
    main()

Level 3:

    # LEVEL 3
    print('LEVEL 3')
    j = 1
    while j < 11:
        solutionA = [1,2,3]
        solutionB = [3,2,2]
        solutionC = [2,1,3]
        solutionD = [2,3,3]
        solutionE = [1,3,2]
        solutionF = [3,1,3]
        solutionG = [3,2,1]
        solutionH = [1,3,3]
        solutionI = [1,1,1]
        solutionJ = [2,3,2]
        i = 0
        response=[]
        print(j)
        k = 0
        sleep(.5)
        while k < j:
            GPIO.output(Correct_LED,GPIO.HIGH)
            GPIO.output(Wrong_LED,GPIO.HIGH)
            sleep(.1)
            GPIO.output(Correct_LED,GPIO.LOW)
            GPIO.output(Wrong_LED,GPIO.LOW)
            sleep(.1)
            k = k + 1
        while True:
            if (response==solutionA and j==1) or (response==solutionB and j==2) or (response==solutionC and j==3) or (response==solutionD and j==4) or (response==solutionE and j==5) or (response==solutionF and j==6) or (response==solutionG and j==7) or (response==solutionH and j==8) or (response==solutionI and j==9) or (response==solutionJ and j==10):
                #print(response)
                print('correct')
                GPIO.output(Correct_LED,GPIO.HIGH)
                sleep(.5)
                GPIO.output(Correct_LED,GPIO.LOW)
                j = j + 1
                break
            elif len(response)==3:
                #print(response)
                print('wrong')
                GPIO.output(Wrong_LED,GPIO.HIGH)
                sleep(.5)
                GPIO.output(Wrong_LED,GPIO.LOW)
                motorG.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                motorY.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                motorR.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                sleep(.5)
                motorG.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                motorY.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                motorR.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                response=[]
            elif killSwitch.is_button_pressed():
                print('Kill switch')
                GPIO.output(Correct_LED,GPIO.HIGH)
                GPIO.output(Wrong_LED,GPIO.HIGH)
                sleep(.5)
                GPIO.output(Correct_LED,GPIO.LOW)
                GPIO.output(Wrong_LED,GPIO.LOW)
                motorG.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                motorY.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                motorR.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                sleep(.5)
                motorG.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                motorY.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                motorR.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                response = []
            else:
                receive( response , i )
    print('LEVEL 3 DONE')

if __name__ == '__main__':
    main()

Level 4:

    # LEVEL 4
    print('LEVEL 4')
    j = 1
    while j < 11:
        solutionA = [1,5]
        solutionB = [2,6]
        solutionC = [2,4]
        solutionD = [3,5]
        solutionE = [3,6]
        solutionF = [1,6,1]
        solutionG = [3,5,2]
        solutionH = [3,4,2]
        solutionI = [1,5,2,3]
        solutionJ = [2,6,3,1]
        i = 0
        response=[]
        print(j)
        k = 0
        sleep(.5)
        while k < j:
            GPIO.output(Correct_LED,GPIO.HIGH)
            GPIO.output(Wrong_LED,GPIO.HIGH)
            sleep(.1)
            GPIO.output(Correct_LED,GPIO.LOW)
            GPIO.output(Wrong_LED,GPIO.LOW)
            sleep(.1)
            k = k + 1
        while True:
            if (response==solutionA and j==1) or (response==solutionB and j==2) or (response==solutionC and j==3) or (response==solutionD and j==4) or (response==solutionE and j==5) or (response==solutionF and j==6) or (response==solutionG and j==7) or (response==solutionH and j==8) or (response==solutionI and j==9) or (response==solutionJ and j==10):
                #print(response)
                print('correct')
                GPIO.output(Correct_LED,GPIO.HIGH)
                sleep(.5)
                GPIO.output(Correct_LED,GPIO.LOW)
                j = j + 1
                break
            elif (len(response)==2 and j<6) or (len(response)==3 and j<9) or len(response)==4:
                #print(response)
                print('wrong')
                GPIO.output(Wrong_LED,GPIO.HIGH)
                sleep(.5)
                GPIO.output(Wrong_LED,GPIO.LOW)
                motorG.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                motorY.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                motorR.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                sleep(.5)
                motorG.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                motorY.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                motorR.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                response=[]
            elif killSwitch.is_button_pressed():
                print('Kill switch')
                GPIO.output(Correct_LED,GPIO.HIGH)
                GPIO.output(Wrong_LED,GPIO.HIGH)
                sleep(.5)
                GPIO.output(Correct_LED,GPIO.LOW)
                GPIO.output(Wrong_LED,GPIO.LOW)
                motorG.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                motorY.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                motorR.run_to_rel_pos(position_sp=-180, speed_sp=400, stop_action="hold")
                sleep(.5)
                motorG.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                motorY.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                motorR.run_to_rel_pos(position_sp=180, speed_sp=400, stop_action="hold")
                response = []
            elif len(response)==1:
                receive2( response , i )
            else:
                receive( response , i )
    print('LEVEL 4 DONE')
    print('YAY WIN!')

if __name__ == '__main__':
    main()