I’m trying to stream audio bytes to a discord bot, but instead of music there’s a lot of noise. Moreover, the bot connects successfully, the codec works. The first thing that comes to mind is that I either take the data in the wrong place, or I transfer it to the bot incorrectly. Moreover, the bot itself works correctly, it connects to the voice channel correctly, and even recognizes sounds, but not the correct ones. I use Exoplayer to play a local mp3 file on the device itself and in Discord. It plays on the phone itself without problems
Discord Bot requrements
To work with discord bots I use Kord. Its requirements for transmitted data are described in the documentation:
A frame of 20ms Opus-encoded 48k stereo audio data.
Codec settings
Based on this, I set the following codec settings, as per the requirement:
@OptIn(UnstableApi::class)
class BotAudioProcessor(
private val audioDataListener: AudioDataListener
): BaseAudioProcessor(){
val codec = Opus()
val SAMPLE_RATE = Constants.SampleRate._48000()
val CHANNELS = Constants.Channels.stereo()
val APPLICATION = Constants.Application.audio()
val FRAME_SIZE = Constants.FrameSize._960()
val COMPLEXITY = Constants.Complexity.instance(10)
val BITRATE = Constants.Bitrate.max()
init {
codec.encoderInit(SAMPLE_RATE, CHANNELS, APPLICATION)
codec.encoderSetComplexity(COMPLEXITY)
codec.encoderSetBitrate(BITRATE)
}
override fun onConfigure(inputFormat: AudioProcessor.AudioFormat): AudioProcessor.AudioFormat {
val outputFormat = AudioProcessor.AudioFormat(
SAMPLE_RATE.v,
CHANNELS.v,
C.ENCODING_PCM_16BIT
)
return outputFormat
}
//... queueInput()
}
Here I set all the parameters as requested in the documentation. All these values were set via the codec, the only thing is that I spent an hour calculating the frame size
frame size = SAMPLE_RATE * 0.02 (20ms) = 960
Getting audio streaming
If I understand correctly, uncompressed PCM streams can be obtained using AudioProcessor, for this, I inherit from BaseAudioProcessor and override the queueInput method, in which I encode the buffer in Opus format, and then pass it to the listener, which will send the stream to the discord bot.
override fun queueInput(inputBuffer: ByteBuffer) {
if (inputBuffer.remaining() < FRAME_SIZE.v / 2) {
Log.e("BotAudioProcessor", "Input buffer too small")
return
}
val frame = ShortArray( FRAME_SIZE.v / 2)
inputBuffer.asShortBuffer().get(frame)
val encoded = codec.encode(frame, FRAME_SIZE)
audioDataListener.onAudioData(encoded!!.toByteArray())
}
Stream data to the bot
Next, the received audio stream is saved in a variable, which is then broadcast by the bot.
@UnstableApi
class MusicBot(...) : AudioDataListener {
private var audioData: ByteArray? = null
//...
@OptIn(KordVoice::class)
suspend fun start() {
kord!!.on<ReadyEvent> {
//...
voiceChannel.connect {
audioProvider {
println(audioBuffer?.joinToString(" "))
println(audioBuffer?.size)
audioBuffer?.let { AudioFrame(it) }
}
}
}
}
override fun onAudioData(data: ByteArray) {
audioBuffer = data
}
}
Links and Additionally
- kord library: https://github.com/kordlib/kord/tree/main/voice
- Opuc codec: https://github.com/theeasiestway/android-opus-codec
- use audioprocessor: https://github.com/google/ExoPlayer/issues/8342
This is how I connect it to the player
@OptIn(UnstableApi::class)
class DiscordRendersFactory
(
context: Context,
private val audioDataListener: AudioDataListener
) : DefaultRenderersFactory(context) {
override fun buildAudioSink(
context: Context,
enableFloatOutput: Boolean,
enableAudioTrackPlaybackParams: Boolean
): AudioSink {
val defaultAudioSink = DefaultAudioSink.Builder()
.setAudioProcessors(arrayOf(BotAudioProcessor(audioDataListener)))
.setEnableFloatOutput(true)
.setEnableAudioTrackPlaybackParams(enableAudioTrackPlaybackParams)
.build()
return defaultAudioSink
}
}
You need to sign in to view this answers
Leave feedback about this