安卓视频处理总结

04-04 16:55:57.099 28015-28195/com.rockylearnstorock.testcamera D/MediaHelper: {csd-1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], mime=video/avc, frame-rate=30, rotation-degrees=270, height=144, width=176, max-input-size=5755, durationUs=21107833, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20]}
04-04 16:55:57.106 28015-28195/com.rockylearnstorock.testcamera D/MediaHelper: {mime=audio/mpeg, durationUs=256835918, encoder-delay=576, channel-count=2, encoder-padding=1681, sample-rate=44100, bit-rate=128000}
04-04 16:55:57.106 28015-28195/com.rockylearnstorock.testcamera E/MPEG4Writer: Unsupported mime ‘audio/mpeg‘
04-04 16:55:57.107 28015-28195/com.rockylearnstorock.testcamera W/System.err: java.lang.IllegalStateException: Failed to add the track to the muxer
04-04 16:55:57.111 28015-28195/com.rockylearnstorock.testcamera W/System.err: at android.media.MediaMuxer.nativeAddTrack(Native Method)
04-04 16:55:57.111 28015-28195/com.rockylearnstorock.testcamera W/System.err: at android.media.MediaMuxer.addTrack(MediaMuxer.java:294)
04-04 16:55:57.112 28015-28195/com.rockylearnstorock.testcamera W/System.err: at com.rockylearnstorock.testcamera.MediaHelper.combineAudioVideo(MediaHelper.java:181)
04-04 16:55:57.112 28015-28195/com.rockylearnstorock.testcamera W/System.err: at com.rockylearnstorock.testcamera.MainActivity$2.run(MainActivity.java:399)
04-04 16:55:57.112 28015-28195/com.rockylearnstorock.testcamera W/System.err: at java.lang.Thread.run(Thread.java:818)

Google "Unsupported mime ‘audio/mpeg‘， 找到StackOverflow： Impossible to mix audio file and video file using MediaMuxer?

里面有个解释是

MediaMuxer does not transcode. If you write out an MPEG4 file, it will expect the video file to be MPEG4/AAC and the audio file to be an AAC file (m4a) as well.

Once you feed it with an m4a, muxing will succeed.

尝试将MP3换成m4a文件，合成，Negtive。

这个贴子里面还有个人发了一个github 例子，尝试， 合成的视频m4a的声音根本就没有加进去。Negtive.

Google " java.lang.IllegalStateException: Failed to add the track to the muxer", 返回的结果，都是" java.lang.IllegalStateException: Failed to stop the track to the muxer"。这个显然不是我要问的。

再看报错行

int audioTrackIndex = muxer.addTrack(audioFormat); //MediaHelper.java:181 报错行

显示是将audioFormat添加到muxer时报错了。

关于MP4和MP3的MediaFormat，打印结果见上面黄色部分：

MP4:{csd-1=java.nio.ByteArrayBuffer[position=0,limit=8,capacity=8], mime=video/avc, frame-rate=30, rotation-degrees=270, height=144, width=176, max-input-size=5755, durationUs=21107833, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20]}

MP3:{mime=audio/mpeg, durationUs=256835918, encoder-delay=576, channel-count=2, encoder-padding=1681, sample-rate=44100, bit-rate=128000}

两者的编码一个是avc, 另一个是mpeg.

根据我的理解， 不同的编码可能造成这个方法报错的原因。因为，“Unsupported mime ‘audio/mpeg‘”。

MediaExtractor audioExtractor = new MediaExtractor();
        try {
            audioExtractor.setDataSource("/storage/emulated/0/DCIM/TestCamera/kisstherain.mp3");
        } catch (IOException e) {
            e.printStackTrace();
            Log.d("MediaHelper", "mp3 file can not be found");
            return;
        }

        MediaFormat audioFormat = null;
        for(int i = 0; i < audioExtractor.getTrackCount(); i++){
            audioFormat = audioExtractor.getTrackFormat(i);
            String mime = audioFormat.getString(MediaFormat.KEY_MIME);
            if(mime.startsWith("audio/")) {
                break;
            }
        }

        if(audioFormat == null){
            Log.d("MediaHelper", "audioFormat is null");
            return;
        }else{
            Log.d("MediaHelper", audioFormat.toString());
        }

        int audioTrackIndex = muxer.addTrack(audioFormat); //MediaHelper.java:181 报错行