Tuesday, December 6, 2011

FFmpeg4Android




FFmpeg4Android is a way your application can run FFmpeg commands,  only Java, no need for C code, or NDK.


=== new ========
Native x86 support!
===============


Download the ffmpeg4android_demo full source code from here:

Download the FFmpeg4Android Library source code from here:


* Note that Demo Project use the FFmpeg4Android Library.

Running FFmpeg4Android demo project (Android Studio)

Android studio 1.0 (and above) makes it very easy, its a simple project import.
If for some reason, you had trouble with it, here is step by step doc with screenshots:


* note that this doc was created with Android Studio 1.0.1

Add FFmpeg support to your app (Android Studio)

This is basically an import module operation.
Download ffmpeg4android_lib (from the link above), and import this module to your project.
If you are not familiar with Android Studio, here is a step by step guide:


Adding FFMpeg4Android support to your app (ADT):


Here is a document that explains how to add FFmpeg4Android support to your app:

This commands verified to work with FFmpeg4Android:

This are only some basic examples, a lot more is possible...


Video Compress:

// simple regular commad
ffmpeg -y -i /sdcard/videokit/in.mp4 -strict experimental -s 160x120 -r 25 -vcodec mpeg4 -b 150k -ab 48000 -ac 2 -ar 22050 /sdcard/videokit/out.mp4

 

// compress with h264 (to support chrome)

ffmpeg -y -i /sdcard/videokit/in.mp4 -strict experimental -vcodec libx264 -preset ultrafast -crf 24 -acodec aac -ar 44100 -ac 2 -b:a 96k -s 320x240 -aspect 4:3 /sdcard/videokit/out3.mp4

// As complex command, don't forget to use setCommandComplex(complexCommand)

// Use this format to support files that contains spaces and special characters

String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/video kit/in.mp4","-strict","experimental","-s", "160x120","-r","25", "-vcodec", "mpeg4", "-b", "150k", "-ab","48000", "-ac", "2", "-ar", "22050", "/sdcard/video kit/out.mp4"};

 

Audio Compression

 

String commandStr = "ffmpeg -y -i /sdcard/vk2/in.wav -ar 44100 -ac 2 -ab 64k -f mp3 /sdcard/videokit/out.mp3";

 

Video Rotate (90 degrees cw):

ffmpeg -y -i /sdcard/videokit/in.mp4 -strict experimental -vf transpose=1 -s 160x120 -r 30 -aspect 4:3 -ab 48000 -ac 2 -ar 22050 -b 2097k /sdcard/video_output/out.mp4

 

Video Crop:

ffmpeg -y -i /sdcard/videokit/short.mp4 -strict experimental -vf crop=100:100:0:0 -s 320x240 -r 15 -aspect 3:4 -ab 12288 -vcodec mpeg4 -b 2097152 -sample_fmt s16 /sdcard/videokit/out.mp4

 

Extract Picture from Video:

ffmpeg -y -i /sdcard/videokit/in.mp4 -strict experimental -an -r 1/2 -ss 00:00:00.000 -t 00:00:03 /sdcard/videokit/filename%03d.jpg

 

Extract Sound from Video:

ffmpeg -y -i /sdcard/videokit/in.avi -strict experimental -acodec copy /sdcard/videokit/out.mp3

ffmpeg -y -i /sdcard/videokit/in.mp4 -strict experimental -vn -ar 44100 -ac 2 -ab 256k -f mp3 /sdcard/videokit/out.mp3

 

Re-encode Audio in Video:

ffmpeg -y -i /sdcard/in.mp4 -strict experimental -vcodec copy -acodec libmp3lame -ab 64k -ac 2 -b 1200000 -ar 22050 /sdcard/out.mp4


Change Video Resolution:

ffmpeg -y -i /sdcard/in.mp4 -strict experimental -vf transpose=3 -s 320x240 -r 15 -aspect 3:4 -ab 12288 -vcodec mpeg4 -b 2097152 -sample_fmt s16 /sdcard/out.mp4


Cut time segment from Video:

ffmpeg -ss 00:00:01.000 -y -i /sdcard/videokit/in.mp4 -strict experimental -t 00:00:02.000 -s 320x240 -r 15 -vcodec mpeg4 -b 2097152 -ab 48000 -ac 2 -b 2097152 -ar 22050 /sdcard/videokit/out.mp4


Transcode Audio:

ffmpeg -y -i /sdcard/videokit/big.wav /sdcard/videokit/small.mp3


WaterMark:

 //  test with watermark.png 128x128, add it to /sdcard/videokit/

String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/videokit/in.mp4","-strict","experimental", "-vf", "movie=/sdcard/videokit/watermark.png [watermark]; [in][watermark] overlay=main_w-overlay_w-10:10 [out]","-s", "320x240","-r", "30", "-b", "15496k", "-vcodec", "mpeg4","-ab", "48000", "-ac", "2", "-ar", "22050", "/sdcard/videokit/out.mp4"}; 


Streaming simple:

 

* note that all streaming examples will need to add internet permission to the Android project Manifest file ( <uses-permission android:name="android.permission.INTERNET" />  )

 
// use this command on ffmpeg4android
ffmpeg -i /sdcard/videokit/2.mpg -strict experimental -f mpegts udp://192.168.0.114:8090

// You can use any player that supports streaming, on the target machine, to play the stream, in this case we used ffplay

ffplay -f mpegts -ast 1 -vst 0 -ar 48000 udp://192.168.0.114:8090 



Stream raw video from camera preview:

  

getting the raw stream from the camera preview:


Parameters parameters = camera.getParameters();
imageFormat = parameters.getPreviewFormat();
if (imageFormat == ImageFormat.NV21) {
                Camera.Size previewSize = parameters.getPreviewSize();
                frameWidth = previewSize.width;
                frameHeight = previewSize.height;
                Rect rect = new Rect(0, 0, frameWidth, frameHeight);
                YuvImage img = new YuvImage(data, ImageFormat.NV21, frameWidth, frameHeight, null);
                try {
                                outStream.write(data);
                                outStream.flush();
                }
}



Encode and stream using FFmpeg4Android: 

"ffmpeg -f rawvideo -pix_fmt nv21 -s 640x480 -r 15 -i " + Environment.getExternalStorageDirectory().getAbsolutePath().toString() + "/yuv.data rtmp://host/stream.flv"


Stream on one device, receive a steam on second device and save it:

on the first device:

ffmpeg -i /sdcard/one3.mp4 -f mpegts udp://192.168.0.107:8090


on the second device:

String[] complexCommand = {"ffmpeg","-y" ,"-i", "udp://192.168.0.108:8090","-strict","experimental","-crf", "30","-preset", "ultrafast", "-acodec", "aac", "-ar", "44100", "-ac", "2", "-b:a", "96k", "-vcodec", "libx264", "-r", "25", "-b:v", "500k", "-f", "flv", "/sdcard/videokit/t.flv"};

* this needs internet permission in the manifest. 

H264 encoding:
ffmpeg -y -i /sdcard/Video/1.MTS -strict experimental -vcodec libx264 -preset ultrafast -crf 24 /sdcard/videokit/out.mp4

ffmpeg -y -i /sdcard/videokit/m.mkv -strict experimental -vcodec libx264 -preset ultrafast -crf 24 -sn /sdcard/videokit/m2.mkv

Subtitles:
ffmpeg -y -i /sdcard/videokit/m2.mkv -i /sdcard/videokit/in.srt -strict experimental -vcodec libx264 -preset ultrafast -crf 24 -scodec copy /sdcard/videokit/mo.mkv

ffmpeg -y -i /sdcard/videokit/m2.mkv -i /sdcard/videokit/in.srt -strict experimental -scodec copy /sdcard/videokit/outm3.mkv

Convert Audio file to m4a
ffmpeg -i /sdcard/videokit/in.mp3 /sdcard/videokit/out.m4a

Encode h264 video and aac audio in one command
ffmpeg -y -i /sdcard/videokit/in.mp4 -strict experimental -vcodec libx264 -crf 24 -acodec aac /sdcard/videokit/out.mkv


Vintage filter

commandStr = "ffmpeg -y -i /sdcard/videokit/in.mp4 -strict experimental -vf curves=vintage -s 640x480 -r 30 -aspect 4:3 -ab 48000 -ac 2 -ar 22050 -b 2097k -vcodec mpeg4 /sdcard/videokit/curve.mp4";

Black & White filter (Gray Scale):

commandStr = "ffmpeg -y -i /sdcard/videokit/in.mp4 -strict experimental -vf hue=s=0 -vcodec mpeg4 -b 2097152 -s 320x240 -r 30 /sdcard/videokit/out.mp4";

Sepia using colorchannelmixer

String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/videokit/sample.mp4","-strict", "experimental", "-filter_complex",
"[0:v]colorchannelmixer=.393:.769:.189:0:.349:.686:.168:0:.272:.534:.131[colorchannelmixed];[colorchannelmixed]eq=1.0:0:1.3:2.4:1.0:1.0:1.0:1.0[color_effect]",
 "-map", "[color_effect]","-map", "0:a", "-vcodec", "mpeg4","-b", "15496k", "-ab", "48000", "-ac", "2", "-ar", "22050","/sdcard/videokit/out.mp4"};

Fade in and out transition
String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/videokit/in.m4v","-acodec", "copy", "-vf", "fade=t=in:st=0:d=5, fade=t=out:st=20:d=5", "/sdcard/videokit/out.mp4"};


Join 2 files using of the same size using filter_complex
String[] complexCommand = {"ffmpeg","-y","-i", "/sdcard/videokit/in1.mp4", "-i", "/sdcard/videokit/in2.mp4", "-strict","experimental", "-filter_complex", "[0:0] [0:1] [1:0] [1:1] concat=n=2:v=1:a=1", "/sdcard/videokit/out.mp4"};



// concat videos with different codecs, and different sizes, different rate and different aspect ratio:

String[] complexCommand = {"ffmpeg","-y","-i","/storage/emulated/0/videokit/sample.mp4",
  "-i","/storage/emulated/0/videokit/in.mp4","-strict","experimental",
  "-filter_complex",
  "[0:v]scale=640x480,setsar=1:1[v0];[1:v]scale=640x480,setsar=1:1[v1];[v0][0:a][v1][1:a] concat=n=2:v=1:a=1",
  "-ab","48000","-ac","2","-ar","22050","-s","640x480","-r","30","-vcodec","mpeg4","-b","2097k","/storage/emulated/0/vk2_out/out.mp4"}


Create a video from pictures

// note that this commans is sensative to the pictures size.
// all pictures should be with the same size, and correspond to s peticular video resolution.
// e.g hd video is: 1280x720, and this should be the pictures size 
commandStr = "ffmpeg -y -r 1/5 -i /sdcard/videokit/pic00%d.jpg /sdcard/videokit/out.mp4";

Advanced filtering
String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/videokit/in.mp4","-strict","experimental", "-vf", "crop=iw/2:ih:0:0,split[tmp],pad=2*iw[left]; [tmp]hflip[right]; [left][right] overlay=W/2", "-vb", "20M", "-r", "23.956", "/sdcard/videokit/out.mp4"};

Increase video and audio speed
String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/videokit/in.mp4","-strict","experimental", "-filter_complex", "[0:v]setpts=0.5*PTS[v];[0:a]atempo=2.0[a]","-map","[v]","-map","[a]", "-b", "2097k","-r","60", "-vcodec", "mpeg4", "/sdcard/videokit/out.mp4"};

Overlay 2 videos side by side
 
     String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/Movies/sample.mp4","-i", "/sdcard/Movies/sample2.mp4", "-strict","experimental",
                "-filter_complex",
                "[0:v:0]pad=iw*2:ih[bg];" +
                "[bg][1:v:0]overlay=w",
                "-s", "320x240","-r", "30", "-b", "15496k", "-vcodec", "mpeg4","-ab", "48000", "-ac", "2", "-ar", "22050",
                "/sdcard/videokit/out.mp4"};  



  vintage + watermark
 
      String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/Movies/sample.mp4","-strict","experimental",
                "-vf",
                "movie=/sdcard/videokit/watermark002.png [watermark];" +
                "[in][watermark] overlay=main_w-overlay_w-10:10 [out_overlay];" +
                "[out_overlay]curves=vintage[out]", 
                "-s", "320x240","-r", "30", "-b", "15496k", "-vcodec", "mpeg4","-ab", "48000", "-ac", "2", "-ar", "22050",
                "/sdcard/videokit/out_water_vinta.mp4"};



Watermark + audio replace
 
    String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/Movies/sample.mp4","-i", "/sdcard/videokit/in.mp3", "-strict","experimental",
                "-filter_complex",
                "[1:a]atempo=1.0[a1];" +
                "movie=/sdcard/videokit/watermark002.png [watermark];" +
                "[0:v][watermark] overlay=main_w-overlay_w-10:10 [outv]",
                "-map", "[outv]", "-map", "[a1]",
                "-s", "320x240","-r", "30", "-b", "15496k", "-vcodec", "mpeg4","-ab", "48000", "-ac", "2", "-ar", "22050",
                "-shortest","/sdcard/videokit/out_water.mp4"};


Looping on a picture and adding audio with audio encoding support
 
      String commandStr = "ffmpeg -y -loop 1 -i /sdcard/videokit/pic001.jpg -i /sdcard/videokit/in.mp3 -strict experimental -s 1270x720 -r 25 -aspect 16:9 -vcodec mpeg4 -vcodec mpeg4 -ab 48000 -ac 2 -b 2097152 -ar 22050 -shortest /sdcard/videokit/out2.mp4";


Replacing video audio track, using the shotest (video or audio)

String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/videokit/sample.mp4","-i", "/sdcard/videokit/in.mp3", "-strict","experimental",
                "-map", "0:v", "-map", "1:a",
                "-s", "320x240","-r", "30", "-b", "15496k", "-vcodec", "mpeg4","-ab", "48000", "-ac", "2", "-ar", "22050","-shortest","/sdcard/videokit/out.mp4"};




Supporting old ARM devices (ARMv5)
Regular FFmpeg4android library is complied for ARMv7 architecture which cover most devices on the market, and gives optimal performance.

In some cases you need to support old devices, and devices that do not comply to ARMv7.
In this case you can use the ARMv5 compilation.

For this reasons, We created FFmpeg4Android for old devices library, and client.


This version does not support complex filtering, and some other advanced FFmpeg capabilities, and
Its significantly slower then the main FFmpeg4Android version.

This version can support both ARMv7 and ARMv5, the device automaticly choose the best fit.
If you need only ARMv5 delete the armeabi-v7a folder from the libs directory.

libs
   armeabi         -> contains libvideokit.so for ARMv5
   armeabi-v7a   -> Remove this folder if you need only ARMv5




Using complex commands

A complex command is a command that contains embedded elements (in most cases it uses quotations for that.
Here is an example:
ffmpeg -i /sdcard/videokit/in.mp4 -aspect 1:1 -vf split "[main][tmp];[tmp] crop=iw/2:ih:0:0, hflip[tp],[tp] pad=2*iw[left]; [main] crop=iw/2:ih:iw/2:0[right]; [left][right] overlay=W/2" -vb 20M -r 23.956 /sdcard/videokit/outs.mp4

To set a complex command you should create a String array and use the method:
setCommandComplex()

Like this:
String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/videokit/mo.mkv","-strict","experimental", "-vf", "crop=iw/2:ih:0:0,split[tmp],pad=2*iw[left]; [tmp]hflip[right]; [left][right] overlay=W/2", "-vb", "20M", "-r", "23.956", "/sdcard/videokit/out.mp4"};

setCommandComplex(complexCommand);


Size Tuning - Keeping your app apk small

FFmpeg4Android lib contains a build specifically to tackle this issue - Small App Size.
This build contains a subset of FFmpeg4Android that contains features (filters, coders, encoders, muxers etc), That are the most commonly used - the "essentials build".
This build is great for apps that need to keep small size.
All the performance benefits of the full ffmpeg4android are kept.

  
To use this build, goto ffmpeg_lib libs/armeabi-v7a folder,
simply rename libvideokit.so to libvideokit.so.full, 
and rename libvideokit.so.ess to libvideokit.so
Refresh the ffmpeg_lib project, and build your project (that use the ffmpeg_lib).
Your project will now use the essentials build, and you will see that the apk created is much smaller in size.

Make sure you test and see that all the functionality you need is still there.

Performance Tuning

  • Prefer mpeg4 codec (-codec mpeg4), since with this codec, ffmpeg4android can utilize all device cores.
Here is an example of command that use mpeg4 codec (in this case this command is doing a video compression):

String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/video kit/in.mp4","-strict","experimental","-s", "160x120","-r","25", "-vcodec", "mpeg4", "-b", "150k", "-ab","48000", "-ac", "2", "-ar", "22050", "/sdcard/video kit/out.mp4"};


  • If its possible, reduce output video resolution (-s) and bitrate (-b), the lower the video quality is, the faster it will take to produce.
  • Encoding using h264 can't utilize all device cores, and thus is much slower. If you must use it, use the -preset ultrafast flag.
  • Using the -threads flag can only degrade the performance, as ffmpeg4android automatically detects the number of cores your device has, and set the optimal number of threads for the command used. 

FFmpeg Command Correctness

FFmpeg command can be quite complex, and in some cases, getting it right can be hard.
To help you with that, FFmpeg4Android will create a file called vk.log that contains the ffmpeg log in the device demo folder (by default /sdcard/videokit) This log can be very useful when understanding what is wrong with the FFmpeg command, and in most cases it will pinpoint you to the section in the command that is wrong.
If you still don't understand what is wrong with your command, make sure the command works on a windows machine, and you can email us the command for inspection.

Adding FFMpeg support to your app (FFmpeg4Android As External Process):

Remove ffmpeg4android app from your device (if you have it installed) and remove the /sdcard/videokit folder.
Add the FFMpeg4Android project library (source code) as a new Android project. 
Download the demo client source which use the FFMpeg4Android library, both are ADT projects.
Run the demo client to your device, you should be able to run the FFMpeg command and rotate the video in.mp4, the output video is /sdcard/videokit/vid_trans.mp4

Full installation document for FFmpeg4Android library and the demo client project.




Latest version release notes


25.23.19
--------
* Improved x264 encoding performance, around 30%
* native x86 support.