3-d Sound!

3-D Sound
Example 8



   Mission :Let's change a little more of our source code to incorporate 3d sound 
            positioning!  Download the Expansion Pack!!

Intro

Today's games require more than just sound playback or even real-time sound mixing. With a whole new breed of high tech games comes a new way to submerse the user into the entire gaming expeirience. The answer is 3-D Sound! I don't know the real math involved in calculating true world 3d sound, but we can easily approximate it in our games. We will first go over the methodology of 3d sound, then we'll move onto the actual code in depth and in your face!

3-d Volumetric Dampening

We are now going to cover the math that will enable us to dampen sounds according to their 3d positions. In order to use 3d sound, we must have a DSP version that supports Stereo mode. If the card doesn't support stereo mode, we can use the 3d distance to dampen the sound, but it won't achieve the awesome 3d effect.

Ok, we first want to know the loudest the sound could be according to its 3d position. We will find this by first calculating the distance from us (0,0,0) to the sound (x,y,z). Distance will be as follows:

Distance = sqrt((x*x)+(y*y)+(z*z));

As you can see, it's only the distance formula for 2d points with the addition of the z extent. Now that we know how far the sound is, we will use that distance as an index in our Cos lookup table. Our lookup table can be however large we want. The bigger it is, the more precise our calculations will be. I decided to make it 500. Our lookup table isn't an ordinary table. We actually have it filled with Cos calculations from 0 to Pi/2. That produces a fall-off curve from 1 to 0 or in our usage from 100% to 0%! Now our sample will be as follows:

sample*= DampTable[Distance];

By doing this multimplication, we are dampening the sound according to how far it is from us. In Mono mode, we will simply send this sample on its way after this little procedure. In Stereo mode, we have 2 channels to program, Left and Right. We now have to figure out which side of us the sound originated from. We only need to check the sign of the x extent of the 3d point to find this out. I decided to use the typical right handed coordinate system which places z positive in front of us, x positive to the right and y positive up. I then refer to the side the sound is on as the Major and the remaining side as the Minor side. Our little figure shows that our sound is originating from the left side.

By now we have our major side calculated, but how do we figure out what to send the other channel? If we set the minor channel the same as the major, we could just as well be in Mono mode! We now have to calculate the Sin of the angle between our left/right to the sound with the x axis being 0 at the right and 180 degrees at the left. Knowing the Sin of this angle, we can dampen the Minor channel according to where it is located according to its x and z extents. I say the Sin of the angle between our left/right because the sin fluxuates between 0 and 1 from 0 to 90 degrees and then it goes from 1 to 0 from 90 to 180 degrees, so the two sides perfectly mirror one another! If you are paying attention, you will realize that we have our dapening equation right at our finger tips! It will be something like this:

sample*= DampTable[Distance];
minorsample = sample* Sin(Theta);

We first use the same calculations that we used for the major side, then dampen that according where the sound is. Close your eyes and imaging a sound directly off your left shoulder. As it circles around you proceding right, your right ear starts to hear more of the sound until both left and right ears hear the same intensity. This will occur when the sound is directly in front of you, or in our case 90 degrees. As it proceeds toward your right, your left ear hears less and less until it hears nothing when the sound is located directly off your right shoulder. This may not be mathmatically perfect, but hey its darn close! Now that we have some idea how we are going to be using the sample data, let's get into the source code to make it happen!

The Code

Let's take a quick look at the header file associated with our SB16 class. The following is a list of fucntions and structures that are new to the class.

typedef struct XYZLocation
{ long x,y,z;
  XYZLocation()
   {x=y=z=0;
   }
};

typedef struct SampleHeader
 {unsigned long Position;  
  unsigned long Length;     
  unsigned short SoundNumber; 
  XYZLocation *Location;     // NEW to tutorial #8
  long OldSample;            //used for stereo mode
  SampleHeader *Next;
  SampleHeader *Previous;
  SampleHeader()
  {Position=Length=SoundNumber=0;
   Next=Previous=NULL;
  }
 }Sampler,*Sample_ptr;

 void Enable3dSound();
 void Disable3dSound();
 long Get3dSampleMono(Sample_ptr S);
 long Get3dSampleStereo(Sample_ptr S);
 long (SB16::*GrabSample)(Sample_ptr);
 void Play(int sound_num,XYZLocation *);
 void CreateDampTable();

 char Sound3d,LeftChannel;
 long *DampTable;

We added a structure to hold 3d coordinates named XYZLocation, added 2 variables to our SampleHeader structure, and added a couple of functions. Sound3d is used as a flag to indicate wether we are in 3d mode or not, 1 and 0 respectively. LeftChannel is used by our mixing routines to tell the rest of the code wether it is programming the left or right channel in stereo mode. DampTable is a long pointer that will be used to hold our dyanmically allocated dampening table. Let's cover some changes made to our existing functions.

SB16::SB16()
{ ...
  Sound3d=0;        //Default non-3d oriented sound
  LeftChannel=0;    //Default starting with left in 3d mixing
  MixingFunction=&SB16::FillBuffer8; //default 8 bit mixing scheme
  GetSample=&SB16::GetSample8;
  ...
}

In our default constructor, we assign 0 to Sound3d and 1 to LeftChannel. This means that we are NOT using 3d sound by default and we are going to start our channel programming with the left side. We also assign two of our 3 function pointers here. Originally we didn't assign them default functions, we simply called SetFunctionPtr() in our setup. Now we no longer have to call SetFunctionPtr at all, the code will change its functions when it needs to! Specifically, they default to 8 bit functions, if we call SetModeBits(..) and pass 16, the code will change to 16 bit mode and call SetPtrFunctions so they are pointing to the correct routines. When we enable 3d sound with Enable3dSound(), the functions will be changed again to work correctly! Let's try to follow the logic of our SetPtrFunctions routine.

void SB16::SetPtrFunctions()
{ if(ModeBits==16)
   {MixingFunction=&SB16::FillBuffer16;
    if(Sound3d)
     {GrabSample=&SB16::GetSample16;        //3d 16 Mono&Stereo
      if(ModeStereoMono)
       {GetSample=&SB16::Get3dSampleStereo; //3d 16 Stereo
       }
      else
       {GetSample=&SB16::Get3dSampleMono;   //3d 16 Mono
       }
     }
    else
     {GetSample=&SB16::GetSample16;         //Non 3d Mono&Stereo
     }
   }
  else
   {MixingFunction=&SB16::FillBuffer8; //default 8 bit mixing scheme
    if(Sound3d)
      {GrabSample=&SB16::GetSample8;        //3d 8 Mono&Stereo
      if(ModeStereoMono)
       {GetSample=&SB16::Get3dSampleStereo; //3d 8 Stereo
       }
      else
       {GetSample=&SB16::Get3dSampleMono;   //3d 8 Mono
       }
     }
     else
      {GetSample=&SB16::GetSample8;
      }
   }
}

Here's where things can get a little confusing! Remember that our GetSample function was used to fetch a sample from our sound. It was important that we knew if we were getting an 8 or 16 bit sample, so it pointed to the correct function depending on what mode we were in. That responsibility rests on GrabSample when in 3d mode. GetSample now must know if is in stereo or mono mode so that we can produce our volumetric dampening in Mono mode or true 3d sound in Stereo mode, this function gets the job done! Now let's look at the functions that really make it all possible!

void SB16::Enable3dSound()
{Sound3d=1;
 CreateDampTable();
 SetPtrFunctions();
}
void SB16::Disable3dSound()
{Sound3d=0;
 SetPtrFunctions();
}

All we need to do to enable 3d sound is create our dampening table, set the correct functions associated with 3d sound, and set our flag to 1! To return to normal sound playback, we set our flag to 0 and reset our function pointers. We don't need to de-allocate the dampening table, we do that in the destructor. If we want to switch to and from 3d sound mode several times, we will have to move the CreateDampTable function to the constructor so we don't have multiple arrays allocated! While we're at it, let's look at that function.

void SB16::CreateDampTable()
{ DampTable = new long[500];
 //making 500 increments of the + end of the Cos wave, from 0 -> Pi/2.
 
 float step= (1.57079632679)/500.0,angle=0.0;
 for(short k=0;k<500;k++)
 {DampTable[k]=(long)(cos(angle)*0x10000);  //angle in radians 0x10000 makes it 16:16 fixed
  angle+=step;
 }
}

Here we allocate 500 longs and fill our array full of cosine values from 0 to Pi/2. This will give us our dampening curve. We multiply that times 0x10000 because our math will be 16:16 fixed point which means no floating point allowed!

void SB16::Play(int sound_num,XYZLocation *P)
{SampleTail->Next = new SampleHeader();
 SampleTail->Next->Previous=SampleTail;
 SampleTail->Next->Next = NULL;
 SampleTail->Next->Position=0;
 SampleTail->Next->Location=P; //NEW!
 SampleTail->Next->Length = Sounds[sound_num].Length;
 SampleTail->Next->SoundNumber = sound_num;
 SampleTail = SampleTail->Next;
}

We created another version of the Play function, this one accepting a 3d point curtesy of the XYZLocation structure. Everything inside is the same except we now set the 3d location variable named ironically 'Location'. I'm so witty :) Let's get into the functions that really make it happen!

//ClipChar Changed
//ClipShort Changed
// Play function changed
//Added Leftchannel^=1 deal to 2 mixing routines above!

long SB16::Get3dSampleMono(Sample_ptr S)
 { long sample;
   double Distance;

   sample=(this->*GrabSample)(S);
   if((S->Location->z > 500)||(S->Location->y >500)||(S->Location->x > 500))
      {S->OldSample=0; //Too far away, return silence!
       return 0;
      }
     Distance=sqrt((S->Location->x*S->Location->x)+
                   (S->Location->y*S->Location->y)+
                   (S->Location->z*S->Location->z));
    if(Distance >500.0)
     {S->OldSample=0; //Still too far away?
      return 0;
     }
    else
     {sample*=DampTable[Distance];
     }
   return sample;
 }

We will be using this function if the program wants 3d sound, but can only support Mono playback. It dampens the sound according to how far away it is. Although the mixing math is 16:16 fixed point, the distance equations are still good old floating point. This is certainly a perfect place to optimize later on! We first test to see if any of the extents are over 500, if this is true, then the distance will certainly be too far away to hear. we then calculate the distance and check again if our distance is too far away to be heard. If everthing passes, we can return the sample dampened by our dampening table with Distance as an index. Let's go on to our Stereo version!

long SB16::Get3dSampleStereo(Sample_ptr S)
{ long sample;
  double Distance,Sin,temp,hyp;

   if(LeftChannel) //get sample : Left Channel
    {sample=(this->*GrabSample)(S);    
     if((S->Location->z > 500)||(S->Location->y >500)||(S->Location->x > 500))
      {S->OldSample=0; 
       return 0;
      }
     Distance=sqrt((S->Location->x*S->Location->x)+
                   (S->Location->y*S->Location->y)+
                   (S->Location->z*S->Location->z));
    if(Distance >500.0)
     {S->OldSample=0;
      return 0;
     }
   
   hyp=sqrt((S->Location->x*S->Location->x)+(S->Location->z*S->Location->z));
   if(hyp == 0)
    {hyp = 1;
    }
   Sin=S->Location->z/hyp;
   if(Sin < 0)
    {Sin*=-1; //We don't want to invert our samples
    }
   
   //ASSERT: Sound is within range so lets start crunching numbers :)
   temp=sample*DampTable[(short)Distance];
   
    if(S->Location->x > 0) //positive x? if so it's to the right of us!
     {/* Sound is to the right*/
      S->OldSample=(long)temp;
      return (long)((double)(temp*Sin));
     }
    else//Sound is to the left of us!
     { S->OldSample=(long)((double)(temp*Sin));
       return (long)temp;
     }

   }
   else //Everything was calculated when we did the left channel!
    {return S->OldSample;
    }
}

Although this function looks pretty big, its functinality follows a very logical path! The entire function is contained in and if with an else. We are going to be setting both the left and right channels, although we can only program them one at a time. This is where the OldSample variable of the Sample_ptr structure comes into play. It holds the value that will be sent off to the right channel. So, if we are programming the left channel right now (LeftChannel != 0) then lets do all the calculations, otherwise let's just return OldSample!
The black text closely resembles what computations we did with the Mono version of this routine. We simply test if any of the extents are too far away. If they are, we will set OldSample to 0 and return 0. This will ensure that both left and right channels return silence. If the points are close enough, we calculate the distance. We then check to see if THAT is too far away. If it is, we do the usual set OldSample to 0 and return 0. By this time, if all passes we will have an index into our dampening table. We now need to figure out which side of us the sound is originating or as i put it in our picture, the Major and Minor sides. This is a very easy task. We only need to look at the sign of the x extent of our 3d point. If it is positive, the sound is to the right of us and vise-versa. We know that we will send our dampened sample to the Major side, but now we need to know how much to dampen the remaining side. We calculate the hypotenuse of a right triangle using the x and z extents. We then use the z extent over the hyp to give us the sin of our viewing angle. We now only need to multiply our dampened sound by this number to get the correct result. The final if-else combination tests which side the sound originates from and programs the channels correctly! You might need to read this part a couple of times. If you understand trig functions, this should be very simple for you! Before we close, let's cover a couple of functions that were changed.

void SB16::FillBuffer8()
{unsigned short i;
 unsigned char *start=(unsigned char*)dma.MK_FP(dma.phys>>4,0);

 SideToFill^=1;
 if(SideToFill)
   { start+=HALFBUFFSIZE;
   }
 for(i=0;i<HALFBUFFSIZE;i++,start++)
  {*start=ClipChar(MixSamples())+128;
    LeftChannel^=1;
  }
}
void SB16::FillBuffer16()
{char *start=(char*)dma.MK_FP(dma.phys>>4,0);
 short sample=0,i;

 SideToFill^=1;
 if(SideToFill)
   { start+=HALFBUFFSIZE;
   }
 for(i=0;i<(HALFBUFFSIZE>>1);i++,start+=2)
  {sample=ClipShort(MixSamples()); //variable clipping function?
   *start=    lobyte(sample);
   *(start+1)=hibyte(sample);
   LeftChannel^=1;
}

The only item added to these two functions are the LeftChannel^=1 commands. This toggles the variable between 0 to 1. This variable is only used in the 3d mixing routines, so the other routines will work fine with these added.

char SB16::ClipChar(long num)
{if(Sound3d)
   { num>>=16;
   }

  if(num > 127)
   {num = 127;
   }
  if(num < -128)
   {num = -128;
   }
  return (char)num;
}
short SB16::ClipShort(long num)
{ if(Sound3d)
   { num>>=16;
   }
  if(num > 32767)
  {num = 32767;
  }
 if(num < -32768)
  {num = -32768;
  }

 return (short)num;
}

You should be wondering how do we convert the samples from 16:16 fixed point to normal! If not, re-read this entire tutorial as a punishment! Remember that our dampening table was multiplied by 0x10000 which aligns it as 16:16 or effectively bitshifting the numbers 16 bits to the left. When we finally go to clip our final mixed sample, we check if we are using 3d sound, and if so, let's shift the result right 16 bits before clipping it!

That's all there is to it! The big secret lies in how we program the seperate channels, but if you did your reading carefully, you'll learn how! If you have any comments, questions, rude remarks, need to pass gas whatever...give me some feedback!!