AudioQueue实现音频流实时播放实战

需求

使用Audio Queue实现实时播放音频流数据.这里以一个装着pcm数据的caf文件为例进行播放.

实现原理

借助数据传输队列,将无论任务数据源的音频数据装入队列中,然后开启audio queue后从队列中循环取出音频数据以进行播放.

阅读前提

Core Audio基本原理: 简书 , 掘金 , 博客
Audio Queue概念篇: 简书 , 掘金 , 博客
Audio Session基础: 简书 , 掘金 , 博客
传输音频数据队列实现
音视频基础知识
C,C++基本知识

博客地址 :Audio Queue Player

总体架构

本例借助队列实现音频数据的中转, 这里用队列是因为audio queue是靠数据驱动以支持播放的,所以有数据回调函数才能持续调用,如果我们不借助队列,就只能在audio queue的类中从回调函数中取来自音频文件的数据,而且假设以后有别的数据源过来,使得音频播放模块代码耦合度越来越高,而这里借助队列的好处是外界不论是音频文件还是音频流仅仅需要放入队列中就好,开启音频模块后我们会从音频队列回调函数中取出队列中的数据,而无需关心数据的来源.

简易流程

AudioStreamBasicDescription
AudioQueueNewOutput
AudioQueueAddPropertyListener
AudioQueueSetParameter
AudioQueueAllocateBuffer
AudioQueueEnqueueBuffer
AudioQueueStart

文件结构

Enm2eer.png!web

快速使用

配置音频数据来源的ASBD

下面是本例中的格式,其他文件需要按文件格式自行配置

// This is only for the testPCM.caf file.
AudioStreamBasicDescription audioFormat = {
    .mSampleRate         = 44100,
    .mFormatID           = kAudioFormatLinearPCM,
    .mChannelsPerFrame   = 1,
    .mFormatFlags        = kLinearPCMFormatFlagIsSignedInteger | kLinearPCMFormatFlagIsPacked,
    .mBitsPerChannel     = 16,
    .mBytesPerPacket     = 2,
    .mBytesPerFrame      = 2,
    .mFramesPerPacket    = 1,
};

配置audio queue player

// Configure Audio Queue Player
[[XDXAudioQueuePlayer getInstance] configureAudioPlayerWithAudioFormat:&audioFormat bufferSize:kXDXReadAudioPacketsNum * audioFormat.mBytesPerPacket];

配置音频文件模块

// Configure Audio File
NSString *filePath = [[NSBundle mainBundle] pathForResource:@"testPCM" ofType:@"caf"];
XDXAudioFileHandler *fileHandler = [XDXAudioFileHandler getInstance];
[fileHandler configurePlayFilePath:filePath];

开始播放

开始播放前先从文件中读取音频数据并放入队列,我们这里先让队列中缓存5帧音频数据,然后再启动audio queue player. 关于音频文件读取以及队列原理这里不做过多说明.如需帮助请参考上文阅读前提.

// Put audio data from audio file into audio data queue
[self putAudioDataIntoDataQueue];

// First put 5 frame audio data to work queue then start audio queue to read it to play.
[NSTimer scheduledTimerWithTimeInterval:0.01 repeats:YES block:^(NSTimer * _Nonnull timer) {
    dispatch_async(dispatch_get_main_queue(), ^{
        XDXCustomQueueProcess *audioBufferQueue = [XDXAudioQueuePlayer getInstance]->_audioBufferQueue;
        int size = audioBufferQueue->GetQueueSize(audioBufferQueue->m_work_queue);
        if (size > 5) {
            [[XDXAudioQueuePlayer getInstance] startAudioPlayer];
            [timer invalidate];
        }
    });
}];

具体实现

1. 定义一个结构体存储音频相关数据

#define kXDXAudioPCMFramesPerPacket 1
#define kXDXAudioPCMBitsPerChannel  16

static const int kNumberBuffers = 3;

struct XDXAudioInfo {
    AudioStreamBasicDescription  mDataFormat;
    AudioQueueRef                mQueue;
    AudioQueueBufferRef          mBuffers[kNumberBuffers];
    int                          mbufferSize;
};
typedef struct XDXAudioInfo *XDXAudioInfoRef;

static XDXAudioInfoRef m_audioInfo;

+ (void)initialize {
    int size = sizeof(XDXAudioInfo);
    m_audioInfo = (XDXAudioInfoRef)malloc(size);
}

2. 初始化

在初始化方法中初始化音频队列,因为本例借助另一个类进行传输,所以这里作为实例对象,以便使用.

- (instancetype)init {
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
        _instace                  = [super init];
        self->_isInitFinish       = NO;
        self->_audioBufferQueue   = new XDXCustomQueueProcess();
    });
    return _instace;
}

3. 配置音频播放器

将传入的音频格式,音频Buffer大小拷贝到实例中

- (void)configureAudioPlayerWithAudioFormat:(AudioStreamBasicDescription *)audioFormat bufferSize:(int)bufferSize {
    memcpy(&m_audioInfo->mDataFormat, audioFormat, sizeof(XDXAudioInfo));
    m_audioInfo->mbufferSize = bufferSize;    
    BOOL isSuccess = [self configureAudioPlayerWithAudioInfo:m_audioInfo
                                                playCallback:PlayAudioDataCallback
                                            listenerCallback:AudioQueuePlayerPropertyListenerProc];
    
    self.isInitFinish = isSuccess;
}

创建audio queue对象实例

通过传入的视频数据格式ASBD, 及回调函数名称即可创建一个对应的audio queue对象.这里将本类作为实例传入,以便回调函数与本类交流.

注意: 因为回调函数是C语言函数的形式,所以无法直接调用类的实例方法.

// Create audio queue
OSStatus status = AudioQueueNewOutput(&audioInfo->mDataFormat,
                                     playCallback,
                                     (__bridge void *)(self),
                                     CFRunLoopGetCurrent(),
                                     kCFRunLoopCommonModes,
                                     0,
                                     &audioInfo->mQueue);

if (status != noErr) {
    NSLog(@"Audio Player: audio queue new output failed status:%d \n",(int)status);
    return NO;
}

监听audio queue工作状态

在回调函数中可以监听audio queue实例工作工作的变化,如正在播放或停止播放.

    // Listen the queue is whether working
    AudioQueueAddPropertyListener (audioInfo->mQueue,
                                   kAudioQueueProperty_IsRunning,
                                   listenerCallback,
                                   (__bridge void *)(self));
                                   
......

static void AudioQueuePlayerPropertyListenerProc  (void *              inUserData,
                                                   AudioQueueRef           inAQ,
                                                   AudioQueuePropertyID    inID) {
    XDXAudioQueuePlayer * instance = (__bridge XDXAudioQueuePlayer *)inUserData;
    UInt32 isRunning = 0;
    UInt32 size = sizeof(isRunning);
    
    if(instance == NULL)
        return ;
    
    OSStatus err = AudioQueueGetProperty (inAQ, kAudioQueueProperty_IsRunning, &isRunning, &size);
    if (err) {
        instance->_isRunning = NO;
    }else {
        instance->_isRunning = isRunning;
    }
    
    NSLog(@"The audio queue work state: %d",instance->_isRunning);
}

验证设置的ASBD音频格式及设置音量

// Get audio ASBD
UInt32 size = sizeof(audioInfo->mDataFormat);
status = AudioQueueGetProperty(audioInfo->mQueue,
                               kAudioQueueProperty_StreamDescription,
                               &audioInfo->mDataFormat,
                               &size);
if (status != noErr) {
    NSLog(@"Audio Player: get ASBD status:%d",(int)status);
    return NO;
}

// Set volume
status = AudioQueueSetParameter(audioInfo->mQueue, kAudioQueueParam_Volume, 1.0);
if (status != noErr) {
    NSLog(@"Audio Player: set volume failed:%d",(int)status);
    return NO;
}

为audio queue buffer分配内存

// Allocate buffer for audio queue buffer
for (int i = 0; i != kNumberBuffers; i++) {
    status = AudioQueueAllocateBuffer(audioInfo->mQueue,
                                      audioInfo->mbufferSize,
                                      &audioInfo->mBuffers[i]);
    if (status != noErr) {
        NSLog(@"Audio Player: Allocate buffer status:%d",(int)status);
    }
}

4. 启动audio queue

预入队几个buffer以驱动播放

因为audio queue是驱动播放的模式,所以只有数据先入队之后才会继续从回调函数中轮循播放,也就是我们需要将前面分配好内存的buffer入队来完成播放.

播放采用从原始音频数据队列中读取音频数据,如下,先出队,然后将音频数据拷贝到 AudioQueueBufferRef 实例,取出需要的信息(此队列仍可继续扩展).

    for (int i = 0; i != kNumberBuffers; i++) {
        [self receiveAudioDataWithAudioQueueBuffer:audioInfo->mBuffers[i]
                                         audioInfo:audioInfo
                                  audioBufferQueue:_audioBufferQueue];
    }
    
    ......
    
- (void)receiveAudioDataWithAudioQueueBuffer:(AudioQueueBufferRef)inBuffer audioInfo:(XDXAudioInfoRef)audioInfo audioBufferQueue:(XDXCustomQueueProcess *)audioBufferQueue {
    XDXCustomQueueNode *node = audioBufferQueue->DeQueue(audioBufferQueue->m_work_queue);
    
    if (node != NULL) {
        if (node->size > 0) {
            UInt32 size = (UInt32)node->size;
            inBuffer->mAudioDataByteSize = size;
            memcpy(inBuffer->mAudioData, node->data, size);
            AudioStreamPacketDescription *packetDesc = (AudioStreamPacketDescription *)node->userData;
            AudioQueueEnqueueBuffer (
                                     audioInfo->mQueue,
                                     inBuffer,
                                     (packetDesc ? size : 0),
                                     packetDesc);

        }

        free(node->data);
        node->data = NULL;
        audioBufferQueue->EnQueue(audioBufferQueue->m_free_queue, node);
    }else {
        AudioQueueStop (
                        audioInfo->mQueue,
                        false
                        );
    }
}

开始工作

OSStatus status;
status = AudioQueueStart(m_audioInfo->mQueue, NULL);
if (status != noErr) {
    NSLog(@"Audio Player: Audio Queue Start failed status:%d \n",(int)status);
    return NO;
}else {
    NSLog(@"Audio Player: Audio Queue Start successful");
    return YES;
}

5. 触发回调函数轮循播放

正如前面所说, audio queue的播放模式是数据驱动式,也就是我们已经预先入队了几个音频队列数据,然后开启audio queue后我们它会自动播放前面已经入队的数据,每当播放完会自动触发回调函数读取数据以完成下一次播放.

static void PlayAudioDataCallback(void * aqData,AudioQueueRef inAQ , AudioQueueBufferRef inBuffer) {
    XDXAudioQueuePlayer *instance = (__bridge XDXAudioQueuePlayer *)aqData;
    if(instance == NULL){
        return;
    }
    
    /* Debug
    static Float64 lastTime = 0;
    NSTimeInterval currentTime = [[NSDate date] timeIntervalSince1970]*1000;
    NSLog(@"Test duration - %f",currentTime - lastTime);
    lastTime = currentTime;
    */
    
    [instance receiveAudioDataWithAudioQueueBuffer:inBuffer
                                         audioInfo:m_audioInfo
                                  audioBufferQueue:instance->_audioBufferQueue];
}

6. 其他

Demo中还有音频队列的暂停, 恢复, 停止, 销毁等功能,较为简单,这里不再说明.

7. 从音频文件中读取音频数据

通过本地文件路径实例化为 CFURLRef 对象

- (void)configurePlayFilePath:(NSString *)filePath {
    char path[256];
    [filePath getCString:path maxLength:sizeof(path) encoding:NSUTF8StringEncoding];
    self->m_playFileURL = CFURLCreateFromFileSystemRepresentation (
                                                                   NULL,
                                                                   (const UInt8 *)path,
                                                                   strlen (path),
                                                                   false
                                                                   );
}

使用时先打开文件

函数中可配置文件权限及类型,本例中文件类型为caf文件.

OSStatus status;
status = AudioFileOpenURL(self->m_playFileURL,
                          kAudioFileReadPermission,
                          kAudioFileCAFType,
                          &self->m_playFile);
if (status != noErr) {
    NSLog(@"open file failed: %d", (int)status);
}

从文件中读取音频数据

首先指定每次读取多少个音频数据包, 该函数会返回最终读取的字节数. 这里通过 m_playCurrentPacket 记录当前读取的音频包数以便下次继续读取.读取完成后关闭文件.

UInt32 bytesRead = 0;
UInt32 numPackets = readPacketsNum;
OSStatus status = AudioFileReadPackets(m_playFile,
                              false,
                              &bytesRead,
                              packetDesc,
                              m_playCurrentPacket,
                              &numPackets,
                              audioDataRef);

if (status != noErr) {
    NSLog(@"read packet failed: %d", (int)status);
}

if (bytesRead > 0) {
    m_playCurrentPacket += numPackets;
}else {
    status = AudioFileClose(m_playFile);
    if (status != noErr) {
        NSLog(@"close file failed: %d", (int)status);
    }
    self.isPlayFileWorking = NO;
    m_playCurrentPacket = 0;
}

需求

实现原理

阅读前提

代码地址 : Audio Queue Player

掘金地址 : Audio Queue Player

简书地址 : Audio Queue Player

博客地址 :Audio Queue Player

总体架构

简易流程

文件结构

快速使用

具体实现

1. 定义一个结构体存储音频相关数据

2. 初始化

3. 配置音频播放器

4. 启动audio queue

5. 触发回调函数轮循播放

6. 其他

7. 从音频文件中读取音频数据

Recommend

垃圾分类逼我去创业，商业模式都想好了

jakartaee-platform/specification at master · eclipse-ee4j/jakartaee-platform · G...

《英雄联盟》什么英雄克制新版「铁男」? - 知乎

一文学会Vue中间件管道

史上最强Java NIO入门：担心从入门到放弃的，请读这篇！

MyBatis基础搭建及架构概述

大数据分析工程师入门（十）：Linux 命令基础

简化 Pod 故障诊断：kubectl-debug 介绍

Matlab加上VBA编程，表格就能画画了

大型互联网系统云原生微服务架构，亿级海量数据治理实践

About Joyk