Technical Audio Questions
Socks
London, UK.Member Posts: 12,822
@CodeWizard @stevej @BlackCloakGS @GeorgeGS
I have a few audio questions, but cannot find clear answers when searching all the usual places . . . . I was wondering if anyone had any input ?
1) On a device like an iPad/iPhone are Music files (m4a's) streamed from disk/EPROM/flash at runtime (rather than loaded into RAM) ?
2) Again sticking with iOS, do stereo Sound files place more of strain on a device's processor at runtime/playback than mono files ? Or are mono files 'padded' to stereo by the device or otherwise played back as dual mono (so making no difference to processor overhead) ?
3) When a Sound is pitched up (let's say 1 octave / '2' on the slider) is the sampling rate doubled ? I mean is - for example - a 22.05kHz sound simply played back at 44.1khz to achieve the pitching or is there some real time resampling going on by the device ?
Thanks for any input !
Comments
Ipads with retina displays have stereo speakers. Iphone just mono.
Guru Video Channel | Lost Oasis Games | FRYING BACON STUDIOS
they are streamed
all sounds are converted to stereo on import. However the overhead should be small that you should not notice a difference in processing if we use mono vs stereo.
I will have to look up in the code what openAl call we make to set the pitch
Curious why sound files have such a massive impact on loading times (though not tested since the optimisations) if they are streamed rather than loaded into RAM?
Contact me for custom work - Expert GS developer with 15 years of GS experience - Skype: armelline.support
ok i looks up in the code and we set the OpenAL AL_PITCH on the source
/**
* Specify the pitch to be applied, either at source,
* or on mixer results, at listener.
* Range: [0.5-2.0]
* Default: 1.0
*/
define AL_PITCH
I believe it's resampling since OpenAL lets you change pitch and speed independently.
Thanks for the input BlackCloakGS, very useful and much appreciated !
I haven't got a clue what that means !
(I don't speak 'code') !!
My question put more simply is, basically, if I am - for example - pitching a 44.1kHz sound up by, let's say, an octave am I asking the target iPhone or iPad to playback the sound at 88.2kHz (to double its pitch) - the reason I ask this is that I have lots (perhaps 15-20) of sounds all firing off at once, and it causes various issues (sounds cutting out mostly - on the device but not in Creator) . . . so I was curious as to whether my pitched up by 1 octave 44.1kHz sound was asking the device to playback lots of high sampling rate (88.2kHz) sounds all at the same time . . . . and this was placing a strain on the device.
. . . and would it be better to author the sound at the right pitch so the device is not having to pitch it up (whatever that involves) in real time ?
. . . . or whether the pitching process does some kind of resampling (for example pitching up an octave by dropping every second sample) so is still playing back at 44.1kHz ?
Hope that makes sense.
My 'are they streamed' question was specifically aimed at Music files rather than Sound files, . . . Sound files are loaded into RAM rather than streamed.
Sound files are not streamed so bigger sound files will increases load times
@Socks have you already verified the sounds will play back through the Music player on the iPad? 88.2kHz is a new sample rate to me. When I google it I see people asking for FLAC/96kHz support, complaining that those sample rates aren't supported by the device in general. But those threads are from like 2013 and earlier, and don't include 5th Gen and on. Could be supported, but that's where my mind goes.
I'm not sure what this question means.
It's simply twice 44.1kHz, so a 44.1kHz sound played an octave higher than its root pitch will be playing at 88.2kHz (on a system supporting 88.2kHz).
I'm not 100% sure what you mean ?
Does the 88.2 kHz sound play on the iPad when using something other than Game Salad to play it?
@socks from what I've read on this they change the speed of playback to increase the pitch. Openal uses a theory called bandlimited interpolation https://ccrma.stanford.edu/~jos/resample/Theory_Ideal_Bandlimited_Interpolation.html
Also each 50% reduction = 1 Octive
Guru Video Channel | Lost Oasis Games | FRYING BACON STUDIOS
When you say 'play on the iPad' do you mean through the speakers ? Or do you mean is an iPad (or whatever) able to generate sound (data) at 88.2khz (for example for a bounced track or for an audio interface) ?
That's kind of what I suspect is happening too.
I guess I mean speakers.
An iPad won't output anymore than 44.1kHz and won't even register a difference between a 16 bit or 24 bit resolution. I still prefer analog tape or vinyl to digital.
Guru Video Channel | Lost Oasis Games | FRYING BACON STUDIOS
https://www.mediafire.com/?2gw9c4je74lcdjy>; @Moik said:
Does a 88.2 kHz sound play on the iPad through the speakers ?
Without a shadow of a doubt no, I own 3 or 4 pairs of good quality (but far from the best) reference monitors that cost upwards of £1500/$2000 each (audio is my day job, my speakers at home are crappy!) and they all top out at under 20kHz, I have some Adam A7Xs that are rated at 42 Hz-50 kHz but they are the exception and are a bit of a moot point as our ears top out around 20kHz (22kHz I think), and even here most adults cannot hear much above 12-14kHz. As a reference, if you were to low-pass a commercial record at 10khz (so there is no information above 10kHz) most people would struggle to notice.
So . . . if we factor in our Nyquist frequency theories we'll see that a 88.2kHz sampling rate will give us a maximum playback rate of 44.1kHz . . . which is more than double what humans are theoretically capable of hearing (in reality most of us can't hear above 15khz, 22.05kHz is the absolute upper upper upper limit) . . . and iPad speakers are tiny compromised mobile device speakers, an educated guess would say their recency response is unlikely to be much more than 150Hz - 15kHz, probably less.
@adent42 @BlackCloakGS
This is reason for my questions (example movie link below if you can be bothered downloading it) - I have a sound that loops while the mouse button is held down / screen is touched - it all works fine, until the audio gets too busy at which point it stops working (this happens around 45 seconds into the movie when the player enters the octaognal area with the big lump of floating mercury above it - up until that point everything is fine, after that point the looped sound stops working, but all other sounds continue to work just fine - this only happens on the device (tested in iPhone 6+ / iPad 3 / iPad Air2), it doesn't happen in Creator.
Of course 'audio gets too busy' is just a theory, but after extensive testing the issue seems to be one of too much audio information (getting rid of certain sounds solves the issues), so I am trying to better understand the audio process to better optimise.
I have tried just about everything I can think off, the sound has been truncated, mono'd, dropped to 48kbps, raised to 128kbps, replaced with a new sound, pitched up, pitched down, you name it, I've probably tried it, but regardless of what I try it cuts out when the general game audio gets too busy.
Example movie: https://www.mediafire.com/?2gw9c4je74lcdjy
You can see at 1':02" that the looped 'shoot' sound is gone ! It actually goes around 0':45.
Ah, so it's intended to have unplayable sounds? I was reading the issue of sounds cutting out as though it was the problem.
@socks can you time it and merge some of them into a single track?
Guru Video Channel | Lost Oasis Games | FRYING BACON STUDIOS
High sampling rates have a bit of an esoteric purpose (and I'm not convinced a valid one), and that's to do with very high frequency distortion, so an iPad (and computer device) could generate something like a 192khz audio file, which you could play back on the right equipment (or even out through your iPad if you have the right kind of digital to analog converter), the idea is that if you get the frequencies outside of our hearing range well recorded it will cause less distortion lower down the frequency range (where we can hear stuff) - there are other advantages to do with mixing and processing - but like I say it's fairly esoteric stuff.
Not 100% sure what that means !?
That's exactly the same solution I arrived at (not tested as yet), the sound is a bullet/shot, so rather than having a short sound that loops, I would merge a bunch of them (say 10 or 20 bullet sounds) into a single audio file and use that, although I would still have to loop it in case you shoot more than 10 or 20 shots (or however long it is) . . . if I can't sort this out merging the sounds is definitely the route I'm going to try next.
@Socks maybe you are hitting the limits of the GS engine ? - current iOS devices using OpenAL allow way more than your 20 or so simultaneous sounds.
Cheers for the input tmann, it's hard to pinpoint the issue, all I know is that it appears to be a sound issue (because switching off other sounds fixes the sound that cuts out during the game) - I'm not sure whether I'm hitting the limits of GS or not, the game is not particularly complex (see movie above) certainly not as complex as other GS games I've made or seen from other users, I'm not using any particularly hi-res images (I think besides the title screen the largest game play element is 256x64px), and I can add and delete a whole bunch of stuff without the frame rate changing (which to me indicates I am operating with the capabilities of GS) . . . ?