37

iOS利用FFmpeg解码音频数据并播放

 5 years ago
source link: https://www.tuicool.com/articles/uAvq6bn
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

需求

利用FFmepg解析并解码音频流数据,然后将解码后的音频数据送给Audio Queue以实现播放.

实现原理

利用FFmpeg解析音频数据流, 利用FFmpeg解码音频数据为PCM格式. 利用Audio Queue Player实现音频数据播放.

阅读前提

代码地址 : Audio Decoder

掘金地址 : Audio Decoder

简书地址 : Audio Decoder

博客地址 :Audio Decoder

总体架构

本例以一个苹果原生相机录制的.MOV文件为例, 将该文件使用FFmpeg解析并解码,将解码后的数据放入传输队列中,然后开启audio queue player, 播放器在回调函数中轮循取出队列中的数据实现播放.

简易流程

FFmpeg parse流程

avformat_alloc_context
avformat_open_input
avformat_find_stream_info
formatContext->streams[i]->codecpar->codec_type == (isVideoStream ? AVMEDIA_TYPE_VIDEO : AVMEDIA_TYPE_AUDIO)
m_formatContext->streams[m_audioStreamIndex]
av_read_frame

FFmpeg解码流程

  • 从parse中的 AVFormatContext 获取音频流对象 AVStream . m_formatContext->streams[m_audioStreamIndex];
  • 获取解码器上下文: formatContext->streams[audioStreamIndex]->codec
  • 获取解码器实例: avcodec_find_decoder(codecContext->codec_id)
  • 打开解码器: avcodec_open2
  • 初始化音频帧: AVFrame *av_frame_alloc(void);
  • 将数据发给解码器: int avcodec_send_packet(AVCodecContext *avctx, const AVPacket *avpkt)
  • 获取解码后的数据: int avcodec_receive_frame(AVCodecContext *avctx, AVFrame *frame);
  • 创建转码器: struct SwrContext *swr_alloc(void);
  • 设置转码器参数: swr_alloc_set_opts
  • 初始化转码器上下文: int swr_init(struct SwrContext *s)
  • 开始转码: int swr_convert(struct SwrContext *s, uint8_t **out, int out_count,const uint8_t **in , int in_count);
  • 获取转码后的data与size.

文件结构

qEBFbiQ.png!web

快速使用

  • 设置音频格式ASBD

    AudioStreamBasicDescription audioFormat = {
    .mSampleRate         = 48000,
    .mFormatID           = kAudioFormatLinearPCM,
    .mChannelsPerFrame   = 2,
    .mFormatFlags        = kLinearPCMFormatFlagIsSignedInteger | kLinearPCMFormatFlagIsPacked,
    .mBitsPerChannel     = 16,
    .mBytesPerPacket     = 4,
    .mBytesPerFrame      = 4,
    .mFramesPerPacket    = 1,
};

  • 配置播放器

    [[XDXAudioQueuePlayer getInstance] configureAudioPlayerWithAudioFormat:&audioFormat bufferSize:kXDXBufferSize];

  • Parse并解码音频文件数据

    - (void)startDecode {
    NSString *path = [[NSBundle mainBundle] pathForResource:@"test" ofType:@"MOV"];
    XDXAVParseHandler *parseHandler = [[XDXAVParseHandler alloc] initWithPath:path];
    XDXFFmpegAudioDecoder *decoder = [[XDXFFmpegAudioDecoder alloc] initWithFormatContext:[parseHandler getFormatContext] audioStreamIndex:[parseHandler getAudioStreamIndex]];
    decoder.delegate = self;
    [parseHandler startParseGetAVPackeWithCompletionHandler:^(BOOL isVideoFrame, BOOL isFinish, AVPacket packet) {
        if (isFinish) {
            [decoder stopDecoder];
            return;
        }

        if (!isVideoFrame) {
            [decoder startDecodeAudioDataWithAVPacket:packet];
        }
    }];
}

  • 获取解码后数据并播放

为了每次能够重新播放,这里需要标记当前是否为解码的第一帧数据,以重新启动播放器. 另一点是使用NSTimer等待音频数据放入队列再开始播放,因为audio queue是驱动播放模式,所以必须等音频数据放入传输队列再开始播放.

#pragma mark - Decode Callback
- (void)getDecodeAudioDataByFFmpeg:(void *)data size:(int)size isFirstFrame:(BOOL)isFirstFrame {
    if (isFirstFrame) {
        dispatch_async(dispatch_get_main_queue(), ^{
            // First put 3 frame audio data to work queue then start audio queue to read it to play.
            [NSTimer scheduledTimerWithTimeInterval:0.01 repeats:YES block:^(NSTimer * _Nonnull timer) {
                
                XDXCustomQueueProcess *audioBufferQueue = [XDXAudioQueuePlayer getInstance]->_audioBufferQueue;
                int size = audioBufferQueue->GetQueueSize(audioBufferQueue->m_work_queue);
                if (size > 3) {
                    dispatch_async(dispatch_get_main_queue(), ^{
                        [[XDXAudioQueuePlayer getInstance] startAudioPlayer];
                    });
                    [timer invalidate];
                }
            }];
        });
    }
    
    // Put audio data from audio file into audio data queue
    [self addBufferToWorkQueueWithAudioData:data size:size];
    
    // control rate
    usleep(16*1000);
}

具体实现

1. 初始化解码器

从Parse模块中可以获取当前文件对应FFmepg的上下文对象 AVFormatContext .因此音频流解码器信息可以直接获取.

  • 获取音频流对象

    AVStream *audioStream = m_formatContext->streams[m_audioStreamIndex];

  • 获取解码器上下文对象

    - (AVCodecContext *)createAudioEncderWithFormatContext:(AVFormatContext *)formatContext stream:(AVStream *)stream audioStreamIndex:(int)audioStreamIndex {
    AVCodecContext *codecContext = formatContext->streams[audioStreamIndex]->codec;
    AVCodec *codec = avcodec_find_decoder(codecContext->codec_id);
    if (!codec) {
        log4cplus_error(kModuleName, "%s: Not find audio codec",__func__);
        return NULL;
    }

    if (avcodec_open2(codecContext, codec, NULL) < 0) {
        log4cplus_error(kModuleName, "%s: Can't open audio codec",__func__);
        return NULL;
    }

    return codecContext;
}

  • 初始化音频帧

AVFrame 作为解码后原始的音视频数据的容器.AVFrame通常被分配一次然后多次重复（例如，单个AVFrame以保持从解码器接收的帧）。在这种情况下，av_frame_unref（）将释放框架所持有的任何引用，并在再次重用之前将其重置为其原始的清理状态。 

// Get audio frame
m_audioFrame = av_frame_alloc();
if (!m_audioFrame) {
    log4cplus_error(kModuleName, "%s: alloc audio frame failed",__func__);
    avcodec_close(m_audioCodecContext);
}

2. 将原始数据发给解码器

调用avcodec_send_packet将压缩数据发送给解码器.最后利用循环接收avcodec_receive_frame解码后的音视频数据.

int result = avcodec_send_packet(audioCodecContext, &packet);
if (result < 0) {
    log4cplus_error(kModuleName, "%s: Send audio data to decoder failed.",__func__);
}

3. 接收解码后的数据. 

result = avcodec_receive_frame(audioCodecContext, audioFrame);

4. 将解码后的数据转码为iOS设备可播放的类型 

 result = avcodec_receive_frame(audioCodecContext, audioFrame);
        while (0 == result) {
            struct SwrContext *au_convert_ctx = swr_alloc();
            au_convert_ctx = swr_alloc_set_opts(au_convert_ctx,
                                                AV_CH_LAYOUT_STEREO,
                                                AV_SAMPLE_FMT_S16,
                                                48000,
                                                audioCodecContext->channel_layout,
                                                audioCodecContext->sample_fmt,
                                                audioCodecContext->sample_rate,
                                                0,
                                                NULL);
            swr_init(au_convert_ctx);
            int out_linesize;
            int out_buffer_size = av_samples_get_buffer_size(&out_linesize,
                                                             audioCodecContext->channels,
                                                             audioCodecContext->frame_size,
                                                             audioCodecContext->sample_fmt,
                                                             1);
            
            uint8_t *out_buffer = (uint8_t *)av_malloc(out_buffer_size);
            // 解码
            swr_convert(au_convert_ctx, &out_buffer, out_linesize, (const uint8_t **)audioFrame->data , audioFrame->nb_samples);
            swr_free(&au_convert_ctx);
            au_convert_ctx = NULL;
            if ([self.delegate respondsToSelector:@selector(getDecodeAudioDataByFFmpeg:size:isFirstFrame:)]) {
                [self.delegate getDecodeAudioDataByFFmpeg:out_buffer size:out_linesize isFirstFrame:m_isFirstFrame];
                m_isFirstFrame=NO;
            }
            
            av_free(out_buffer);
        }
        
        if (result != 0) {
            log4cplus_error(kModuleName, "%s: Decode finish.",__func__);
        }
`

report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK