Tuesday, December 6, 2011

FFmpeg4Android

FFmpeg4Android is a way your application can run FFmpeg commands,  only Java, no need for C code, or NDK.

 FFmpeg4Android is now using FFmpeg 2.0 as its code base.

Getting the Library and running the Demo App


1. Download FFmpgeg4Android from the Android market, and install on your device, and test your commands (Note that complex commands could not be tested using this app). 





Adding FFMpeg support to your app (Full FFmpeg4Android):

Remove ffmpeg4android app from your device, and remove the /sdcard/videokit folder.
Add the FFMpeg4Android project library (source code) as a new Android project. 
Download the demo client source which use the FFMpeg4Android library, both are ADT projects.
Run the demo client to your device, you should be able to run the FFMpeg command and rotate the video in.mp4, the output video is /sdcard/videokit/vid_trans.mp4

Full installation document for FFmpeg4Android library and the demo client project.


Quick Overview for Adding support to your app

* Writing your own app is extremely simple, all you need is to follow the "demo client" project lead, here are the steps:
1. Add the FFmpeg4Android library to your app.
2. Add the service and the needed permissions to your manifest:

     <service android:name="com.netcompss.ffmpeg4android.FFMpegRemoteServiceBridge" 
          android:process=":remote"
          android:exported="true">
              <intent-filter><action android:name="com.netcompss.ffmpeg4android.FFMpegRemoteServiceBridge"></action></intent-filter>
        </service>


<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>

<uses-permission android:name="android.permission.WAKE_LOCK" /> 

2. Like in demo app, create an Activity that extends BaseWizard.
Now you can call the transcoding like this:

remoteService.setFfmpegCommand(command);
runWithCommand(command);  

Command is a simple String (please note that you are responsible for the validation of this String as a valid FFmpeg command)

That's it!
Your app has FFmpeg support


Adding FFMpeg support to your app (Direct JNI FFmpeg4Android):

Direct JNI FFmpeg4Android is the mean and lean version of FFmpeg4Android, basicly few lines of code,
and call to the JNI interface.
This is the most strait forward way to go if you want maximum flexability to your app.
This way, FFmpeg commands can be called from your app, from every type of Android Object, or Java Object e.g Service, Fragment, Activity, Thread etc, simply call the JNI directly.

Another advantage is that the native interface is optimized for size and its size if around 5M inside the APK.

Download the ffmpeg4android_client_demo_native full source code from here.


 Commands Examples

This commands verified to work with FFMpeg4Android:

This are only some basic examples, a lot more is possible...


Video Compress:

ffmpeg -y -i /sdcard/videokit/in.mp4  -strict experimental -s 160x120 -r 25 -aspect 3:4 -vcodec mpeg4 -b 2097152 -ab 48000 -ac 2 -ar 22050 /sdcard/videokit/out.mp4


Video Rotate (90 degrees cw):

ffmpeg -y -i /sdcard/videokit/in.mp4 -strict experimental -vf transpose=1 -s 160x120 -r 30 -aspect 4:3 -ab 48000 -ac 2 -ar 22050 -b 2097k /sdcard/video_output/out.mp4


Video Crop:

ffmpeg -y -i /sdcard/videokit/short.mp4 -strict experimental -vf crop=100:100:0:0 -s 320x240 -r 15 -aspect 3:4 -ab 12288 -vcodec mpeg4 -b 2097152 -sample_fmt s16 /sdcard/videokit/out.mp4 


Extract Picture from Video:

ffmpeg -y -i /sdcard/videokit/in.mp4 -strict experimental -an -r 1/2 -ss 00:00:00.000 -t 00:00:03 /sdcard/videokit/filename%03d.jpg


Extract Sound from Video:

ffmpeg -y -i /sdcard/videokit/in.avi -strict experimental -acodec copy /sdcard/videokit/out.mp3


Re-encode Audio in Video:

ffmpeg -y -i /sdcard/in.mp4 -strict experimental -vcodec copy -acodec libmp3lame -ab 64k -ac 2 -b 1200000 -ar 22050 /sdcard/out.mp4


Change Video Resolution:

ffmpeg -y -i /sdcard/in.mp4 -strict experimental -vf transpose=3 -s 320x240 -r 15 -aspect 3:4 -ab 12288 -vcodec mpeg4 -b 2097152 -sample_fmt s16 /sdcard/out.mp4


Cut time segment from Video:

ffmpeg -y -i 101_0077.MP4 -strict experimental -vcodec copy -acodec copy -ss 00:00:00 -t 00:00:01 out.mp4


Transcode Audio:

ffmpeg -y -i /sdcard/videokit/big.wav /sdcard/videokit/small.mp3


Streaming simple:

 

* note that all streaming examples will need to add internet permission to the Android project Manifest file ( <uses-permission android:name="android.permission.INTERNET" />  )

 
// use this command on ffmpeg4android
ffmpeg -i /sdcard/videokit/2.mpg -strict experimental -f mpegts udp://192.168.0.114:8090

// You can use any player that supports streaming, on the target machine, to play the stream, in this case we used ffplay

ffplay -f mpegts -ast 1 -vst 0 -ar 48000 udp://192.168.0.114:8090 


Stream raw video from camera preview:

"ffmpeg -f rawvideo -pix_fmt nv21 -s 640x480 -r 15 -i " + Environment.getExternalStorageDirectory().getAbsolutePath().toString() + "/yuv.data rtmp://host/stream.flv"


Stream on one device, receive a steam on second device and save it:

on the first device:

ffmpeg -i /sdcard/one3.mp4 -f mpegts udp://192.168.0.107:8090


on the second device:

String[] complexCommand = {"ffmpeg","-y" ,"-i", "udp://192.168.0.108:8090","-strict","experimental","-crf", "30","-preset", "ultrafast", "-acodec", "aac", "-ar", "44100", "-ac", "2", "-b:a", "96k", "-vcodec", "libx264", "-r", "25", "-b:v", "500k", "-f", "flv", "/sdcard/videokit/t.flv"};

* this needs internet permission in the manifest. 

H264 encoding:
ffmpeg -y -i /sdcard/Video/1.MTS -strict experimental -vcodec libx264 -preset ultrafast -crf 24 /sdcard/videokit/out.mp4

ffmpeg -y -i /sdcard/videokit/m.mkv -strict experimental -vcodec libx264 -preset ultrafast -crf 24 -sn /sdcard/videokit/m2.mkv

Subtitles:
ffmpeg -y -i /sdcard/videokit/m2.mkv -i /sdcard/videokit/in.srt -strict experimental -vcodec libx264 -preset ultrafast -crf 24 -scodec copy /sdcard/videokit/mo.mkv

ffmpeg -y -i /sdcard/videokit/m2.mkv -i /sdcard/videokit/in.srt -strict experimental -scodec copy /sdcard/videokit/outm3.mkv

Convert Audio file to m4a
ffmpeg -i /sdcard/videokit/in.mp3 /sdcard/videokit/out.m4a

Encode h264 video and aac audio in one command
ffmpeg -y -i /sdcard/videokit/in.mp4 -strict experimental -vcodec libx264 -crf 24 -acodec aac /sdcard/videokit/out.mkv


Vintage filter

commandStr = "ffmpeg -y -i /sdcard/videokit/in.mp4 -strict experimental -vf curves=vintage -s 640x480 -r 30 -aspect 4:3 -ab 48000 -ac 2 -ar 22050 -b 2097k -vcodec mpeg4 /sdcard/videokit/curve.mp4";

Fade in and out transition
String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/videokit/in.m4v","-acodec", "copy", "-vf", "fade=t=in:st=0:d=5, fade=t=out:st=20:d=5", "/sdcard/videokit/out.mp4"};


Join 2 files using of the same size using filter_complex
String[] complexCommand = {"ffmpeg","-y","-i", "/sdcard/videokit/in1.mp4", "-i", "/sdcard/videokit/in2.mp4", "-strict","experimental", "-filter_complex", "[0:0] [0:1] [1:0] [1:1] concat=n=2:v=1:a=1", "/sdcard/videokit/out.mp4"};



// concat videos with different codecs, and different sizes, different rate (limitation, needs the same aspect on both videos):

String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/videokit/in.mp4","-i", "/sdcard/videokit/in.mp4","-strict","experimental","-filter_complex", "[0:v]scale=w=160:h=120[v1]; [1:v]scale=w=160:h=120[v2]; [v1][0:a][v2][1:a] concat=n=2:v=1:a=1 [v] [a]","-map","[v]","-map","[a]", "-b", "2097k", "-vcodec", "mpeg4", "/sdcard/videokit/out.mp4"};


Create a video from pictures
commandStr = "ffmpeg -y -r 1/5 -i /sdcard/videokit/pic00%d.jpg /sdcard/videokit/out.mp4";

Advanced filtering
String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/videokit/in.mp4","-strict","experimental", "-vf", "crop=iw/2:ih:0:0,split[tmp],pad=2*iw[left]; [tmp]hflip[right]; [left][right] overlay=W/2", "-vb", "20M", "-r", "23.956", "/sdcard/videokit/out.mp4"};

Increase video and audio speed
String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/videokit/in.mp4","-strict","experimental", "-filter_complex", "[0:v]setpts=0.5*PTS[v];[0:a]atempo=2.0[a]","-map","[v]","-map","[a]", "-b", "2097k","-r","60", "-vcodec", "mpeg4", "/sdcard/videokit/out.mp4"};

Supporting all ARM based devices
FFmpeg4android native library is complied for ARMv7 architecture which cover most devices on the market, and gives optimal performance.
In some cases you need to support old devices, and devices that do not comply to ARMv7.
In this case you can use the ARMv5 compilation.
Note that using ARMv5 can support more devices but its much slower then ARMv7.
To use it you should download our extra ARM libs and replace the libvideokit.so in the original FFmpeg4Android.

It also possible to include both and let the device choose the best fit, in this case you should use this annotation:

libs
   armeabi         -> put the armv5 libvideokit.so
   armeabi-v7a   -> use the original libvideokit.so

* The downside of this approach is that ffmpeg4android library is twice the size, and its possible some devices will not choose the correct libvideokit.so 


Using complex commands
A complex command is a command that contains embeded elements (in most cases it uses quotations for that.
Here is an example:
ffmpeg -i /sdcard/videokit/in.mp4 -aspect 1:1 -vf split "[main][tmp];[tmp] crop=iw/2:ih:0:0, hflip[tp],[tp] pad=2*iw[left]; [main] crop=iw/2:ih:iw/2:0[right]; [left][right] overlay=W/2" -vb 20M -r 23.956 /sdcard/videokit/outs.mp4

To set a complex command you should create a String array and use the method:
setCommandComplex()

Like this:
String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/videokit/mo.mkv","-strict","experimental", "-vf", "crop=iw/2:ih:0:0,split[tmp],pad=2*iw[left]; [tmp]hflip[right]; [left][right] overlay=W/2", "-vb", "20M", "-r", "23.956", "/sdcard/videokit/out.mp4"};

setCommandComplex(complexCommand);




The internals 
As mentioned before, FFmpeg4Android runs as an Android remote service, this service is actually a separate Android process, that encapsulate the FFmpeg native process, that runs directly on the device Linux.
The demo application command will convert vid.mp4 to vid_trans.mp4, it will rotate it 270 degrees, change its resolution to 320x240, change the aspect ratio to 3:4 and encode it with mpeg4 encoder, this is all with one command! (actually it does more, it defines the audio codec, and set the audio bit rate also). 
This is done at the native FFmpeg level, JNI is used to make the calls from the Remote service to the native FFmpeg process.

Updates - new version - 8.0
* Version 8 is a significant upgrade, Its based on FFmpeg 2.0 and can now support much more complex commands.

Here is a command example:
String[] complexCommand = {"ffmpeg","-y" ,"-i", "/sdcard/videokit/mo.mkv","-strict","experimental", "-vf", "crop=iw/2:ih:0:0,split[tmp],pad=2*iw[left]; [tmp]hflip[right]; [left][right] overlay=W/2", "-vb", "20M", "-r", "23.956", "/sdcard/videokit/out.mp4"};


Updates - new version - 7.0 

* Version 7 simplifies the client interface, 2 java lines of code to run transcoding via you app.

* Added simple customization options via optional commands, for example setting the notification icon.
Here are the optional commands:




setProgressDialogTitle("Transcoding...");

setProgressDialogMessage(" transcoding can take a few minutes");

setNotificationIcon(R.drawable.notification_icon);

setNotificationMessage("Demo is running...");

setNotificationTitle("DemoClient");

setNotificationfinishedMessage("Transcoding finished");

setNotificationStoppedMessage("Transcoding stopped");



Using this methods from your application, you can easily and quickly change the FFmpeg4Android messages, and notifications.


Updates - new version - 6.0 
* Version 6 now supports simplified library installation.
* No need for apk installation to run the demo client, simply add the FFmpeg4Android library to your project, and you are good to go.

Updates - new version - 5.0 
* Version 5 now support Background processing out of the box, no need to write any code for that!

*  Version 5 has streaming support,  to stream use this command: 
String command = "ffmpeg -i /sdcard/videokit/2.mpg -f mpegts udp://192.168.0.114:8090";
To get the streamed video use this on the machine you stream to (in this case 192.168.0.114):
ffplay -f mpegts -ast 1 -vst 0 -ar 48000 udp://192.168.0.114:8090

Simplified api, all your app has to do is add the library, and with a few lines of code you have FFMpeg support

 Updates - new version - 4.0 
This use the latest FFmpeg stable source to date, and adds libmp3lame to support mp3 encoding, so now you can easily run commands like this:
 * ffmpeg -y -i /sdcard/videokit/in.wav out.mp3
This command transcode WAV file to MP3 file
* ffmpeg -y -i /sdcard/videokit/in.mp4 -vn -ar 44100 -ac 2 -ab 192 -f mp3 /sdcard/videokit/out.mp3
This command extract mp3 sound from the input video.