Flying the AR.Drone 2.0 with Windows Store Applications

[EDN Admin]

The AR.Drone 2.0

The Parrot AR.Drone 2.0 is an awesome device packed with cool features. The drone contains two cameras, one pointed forward that streams live video, and one pointed downwards (for all your surveillance needs). Its got four powerful engines that make it fast and maneuverable. The drone’s firmware keeps it stable and level while stationary or performing maneuvers, handling a huge burden for the user. It’s a fun device to write code for, and even more fun to pilot when you’re done!
There’s a few projects that have created AR.Drone APIs that can be used for Windows 8 Desktop Apps. However, I wanted to fly my drone from a Windows Store App, so I decided to build my own API compatible with WinRT. The result is covered in this series of posts. Thanks to Nisha Singh (http://blogs.msdn.com/b/nishasingh/) and everyone else who helped me out with this project!
In the first post, I’ll go over how the API handles communicating with and controlling the drone, and then demonstrate how to use the API to make a simple control App.

API Information and Terms

The API consists of 5 files:

• DroneConnection.cs: functionality to connect to, and disconnect from, the drone.
• DroneControl.cs: contains the control loop.
• DroneMovement.cs: contains methods that construct and return fully-formatted commands.
• InputProcessing.cs: determines the next command to issue to the drone.
• InputState.cs: used to pass information between the UI and back-end.
• 
  The terms the API code uses to refer to axes of movement are as follows:
• DroneStrafeX: left-right.
• DroneStrafeY: forward-backward.
• DroneRotateX: yaw, or rotation.
• DroneAscendY: up-down.
• 
  
  Communicating with the Drone
  
  The AR.Drone 2.0 communicates with other devices through its own wireless network. Sending the drone instructions is as simple as connecting your computer to the drone’s network, opening a UDP connection to the drone (I.P. 192.168.1.1, Port 5556), and then sending commands over this connection. The code to connect to the drone is contained inside DroneConnection.cs:
  // Initialize the connection to the drone
  public static async Task ConnectToDrone()
  {
  
  ...
  
  // Set up the UDP connection
  string remotePort = "5556";
  HostName droneIP = new HostName("192.168.1.1");
  
  udpSocket = new DatagramSocket();
  await udpSocket.BindServiceNameAsync(remotePort);
  await udpSocket.ConnectAsync(droneIP, remotePort);
  udpWriter = new DataWriter(udpSocket.OutputStream);
  
  ...
  
  }
  
  
  Once a connection has been established, we can begin to send the drone messages:
  
  // Send a command to the drone
  public static async Task SendDroneCommand(string command)
  {
  udpWriter.WriteString(command);
  await udpWriter.StoreAsync();
  }
  
  This networking code is discussed more in-depth here: Try, Catch, Finally... - UDP and Windows 8 Apps
  The drone needs to continuously receive commands for it to remain responsive. If the drone doesn’t receive any commands for two seconds, it assumes the connection has been lost, and it will ignore any further commands. If this happens, you need to close and re-establish the connection before you can continue to pilot the drone. DroneConnection.cs contains methods that allow you to do this.
  
  Commanding the Drone
  
  You can command the drone by sending it strings called AT commands. There are seven different categories of AT commands, but only two are required to fly the drone, and are the only ones included in the API:
• AT*REF – Used for basic behavior such as takeoff/landing, and emergency stop/reset.
• AT*PCMD – Used to fly the drone once airborne. This controls pitch, yaw, roll, and altitude.
• 
  AT commands have very specific formatting, and consist of four parts. An improperly formatted command will be ignored by the drone. The different sections of an AT*PCMD command are highlighted below:
  
  AT*PCMD=23,1,0,0,0,0rn
  
  Header: The type of the AT command
  Sequence Number: The sequence number is used by the drone to keep track of which commands it needs to execute. The drone will not execute any commands with a sequence number lower than the highest sequence number it has received so far. The first command sent should always have a sequence number of 1, and each command sent subsequently should increment this number by 1. The sequence number can be manually reset by sending a command with the number 1.
  Payload: This carries the command arguments. Its format and content vary from command to command. In the AT*PCMD command, the first value is a flag indicating whether the drone should do nothing (low) or move according to the other arguments (high). The next four values denote roll, pitch, vertical speed, and angular speed respectively. These values are floats ranging from [-1, 1], where (0, 1] represents movement speed in one direction, and [-1, 0) represents the other. A 0 causes the drone to remain stationary along that axis, a 1 or –1 is maximum movement speed in their respective directions, and each value between is a fraction of the maximum possible speed. For example, in the pitch argument, a .75 would cause the drone to move backward at 3/4 maximum speed, while a –.5 would make it move forward at 1/2 the possible speed. However, the drone doesn’t respond to values in the [-1, 1] range. The actual value that needs to be inserted into the AT command is the signed integer value represented by the bytes the float is actually stored in. This conversion is done for you in the FloatConversion method in DroneMovement.cs.
  Carriage Character: A newline character. Use the one specific to your environment.
  DroneMovement.cs has a set of methods to generate AT*REF and AT*PCMD commands, all of which take the current sequence number as an argument, and return a ready-to-send AT command. Some methods take additional arguments to control the movement speed. An example of one of these methods is:
  // Strafe drone forward or backward at velocity in range [-1,1]
  public static string GetDroneStrafeForwardBackward(uint sequenceNumber, double velocity)
  {
  // Convert the ratio into a value the drone understands
  int value = FloatConversion(velocity);
  return CreateATPCMDCommand(sequenceNumber, "0," + value + ",0,0");
  }
  
  // Return a full ATPCMD command
  private static string CreateATPCMDCommand(uint sequenceNumber, string command)
  {
  return "AT*PCMD=" + sequenceNumber + ",1," + command + Environment.NewLine;
  }
  
  Control Infrastructure
  The drone is controlled with an continuous loop that sends a command to the drone every 30 ms. Each cycle, the loop determines the next command, sends the command to the drone, and then increments the sequence number. The method is asynchronous and awaits the call to System.Threading.Tasks.Task.Delay() so that it doesn’t block the UI between cycles.
  
  public static async void ControlLoop()
  {
  while (true)
  {
  
  ...
  
  // Get and send the next command
  string commandToSend = InputProcessing.NextCommand(sequenceNumber);
  await DroneConnection.SendDroneCommand(commandToSend);
  
  sequenceNumber++;
  
  await System.Threading.Tasks.Task.Delay(30);
  }
  }
  
  You need to implement the InputProcessing.NextCommand() method to return an AT command based on criteria of your own choosing. The API includes the class InputState.cs that you can use to pass information between your UI and the control infrastructure. The example App contains an example of this.
  Example App
  
  I’ve included a simple example App to demonstrate how to use the API. The App allows you to connect to the drone and make it take off, land, and rotate. It also illustrates how to pass input information between the UI and the control infrastructure using InputState instances. In the App, FlyingPage contains the method GetState() that returns an instance of InputState. GetState() looks at the input the App is receiving, then creates and returns an instance of InputState containing that information.
  // returns the current state of the input controls
  public static InputState GetState()
  {
  // slider value is in the range [-1,1]
  return new InputState(0, 0, 0, sliderValue, isFlying);
  }
  
  InputProcessing.NextCommand() calls GetState, and then interprets the state to decide on the next command to send:
  // returns the next command that should be executed
  public static string NextCommand(uint sequenceNumber)
  {
  InputState state = MainPage.GetState();
  
  // Determine if the drone needs to take off or land
  if (state.startFlying && !isFlying)
  {
  isFlying = !isFlying;
  return DroneMovement.GetDroneTakeoff(sequenceNumber);
  }
  else if (!state.startFlying && isFlying)
  {
  isFlying = !isFlying;
  return DroneMovement.GetDroneLand(sequenceNumber);
  }
  
  
  // Check if the drone needs to rotate
  if (state.rotateX != 0)
  {
  return DroneMovement.GetDroneRotate(sequenceNumber, state.rotateX);
  }
  
  // Otherwise have the drone do nothing.
  return DroneMovement.NullCommand(sequenceNumber);
  }
  
  The command decision is made in order of importance. Takeoff and landing commands are the most important, followed by movement commands. If there is no other command to send, NextCommand() returns a null command so that the drone will continue hovering and listening to the connection.
  Once NextCommand() has been implemented, all the App needs to do is connect to the drone and initiate the control loop by calling DroneConnection.ConnectToDrone(), followed by calling DroneControl.ControlLoop(). In the example App, this is done in the handler for the connect button. Once the control loop is running, the drone is ready to fly!
  Get Flying
  
  The API should give you everything you need to get started writing drone Apps for Windows Store. In part 2, I’ll discuss a more complete App I’ve written to pilot the drone. There’s still so much drone functionality to be exposed, so get coding and get flying!

View the full article