I have finally managed to get the camera to work! I control the vertical! I control the horizontal!
Ok, so the mouse now moves the camera around. After days and hours of experimentation, I have figured out how it all fits together. I am making a 6DOF camera, although right now only attitude and bearing are affected (pitch and yaw).
The way this works is that the camera stores three vectors: the position of the camera, the view, and the up vector. Position is fairly straight forward. View is the direction in which the camera is looking. Up vector serves to orient the camera, indicating which way is up. All pretty self explanatory.
I then use the gluLookAt function to set the correct perspective. One warning about the way I do this, my view vector stores the direction the camera is facing, but gluLookAt expects as one of its parameters the position of something you're looking at, which is then used as the center of the screen. The way I get this position is to add the camera's position and the view vector and passing this along to gluLookAt.
gluLookAt then subtracts the position from the target to get its own view vector which it then normalizes. So gluLookAt is undoing some of my work. I will do a function of my own to do what gluLookAt does, so I avoid the unnecessary step, but it isn't a priority at this time.
The trick is how to get the camera to rotate, with what I have. The solution I am using is using quaternions to describe the rotations. The way this works is by incrementally rotating the view and up vectors with the quaternions.
I use the SDL to detect when the mouse moves, which creates an event. This provides me with both absolute and relative movement. Absolute movement provides the screen coordinates of the mouse, with (0,0) being the top left corner. Relative movement is the accumulated movement since the last event. Because the program doesn't keep track of the mouse outside of the screen, I have to warp the mouse back to the center of the screen each time it moves away.
This was creating a problem where the warp movement created a mouse movement event that was the opposite of the movement itself,. To fix it I am now checking the absolute position of the mouse when I catch an event, and if it is at the center I ignore that particular event. Since it can only be at the center right after warping, and any movement I care about draws it away, I don't lose anything.
Once I have the relative movement, I multiply it by a sensitivity factor and store it as the number of degrees the camera moved in that frame. Horizontal movement of the mouse affects the bearing of the camera, while vertical movement affects the attitude.
So, on to the rotations themselves. Changes in bearing are described as a rotation around the up vector of the camera. This modifies the view vector, but leaves position and up unchanged. Quaternions are used here to describe this rotation. I use a convert_axis_angle function, using the up vector and the angle to build the quaternion and then normalize it. I then create a quaternion from my view vector, which for any given vector v = [ x , y , z ] , q = [ 0 , v ] .
The rotation of the view vector is the sandwich product qbearing x qview x conjugate(qbearing) . This result is stored in view.
For attitude changes, I need to determine the cross product of view and up, which gives me the right vector of the coordinate trio ( view , up , right ). This time, both view and up are modified by the rotation. So I create the attitude quaternion with convert_axis_angle, using the right vector this time, and make both rotations:
qattitude x qview x conjugate(qattitude)
qattitude x qup x conjugate(qattitude)
The results are stored in view and up respectively.
After this is done, I use gluLookAt as described above, and voila, it is done.
There's room for growth, of course. Roll can be easily appended, once the input is mapped. In the case of roll, the rotation would be around the view vector, and only up would be modified. I also need to add translation, to move the camera around. This is easily done, however, now that I have a view vector. Lateral movement (strafing) is done by getting the right product of view and up, and moving along that vector, vertical movement is done along the up vector and forward movement is done along the view vector.
Ok, that's it for now. Next up, making the world a more interesting place.