Tuesday, December 6, 2011

FFmpeg4Android

FFmpeg4Android is a way your application can run FFmpeg commands,  only Java, no need for C code, or NDK.

Getting the Library and running the Demo App

1. Download FFmpgeg4Android from the Android market, and install on your device, and test your commands. 







2. To add FFMpeg support to your app:

 2.1  Add the FFMpeg4Android project library (source code) as a new Android project. you can also download the demo client source which use the FFMpeg4Android library, both are ADT projects, simply compile and run them.

 2.2 To bundle it with your app, You need to use the FFMpeg4Android project library (source code), (the demo client source is good only if you have the FFmpgeg4Android apk installed, which is not the case if you want to bundle it with your app).
In the ffmpeg4android source code you have the class com.example.DemoClient as a place holder for your code, and you can simply use it as is to run your code (with few renames and refactors).
Put your FFMpeg command in the invokeService():

remoteService.setFfmpegCommand(command);
The command is a simple String.

then, run using:
 runWithCommand(command);

Thats it!

 Commands Examples

This commands verified to work with FFMpeg4Android:

This are only some basic examples, a lot more is possible...

Video Compress:

ffmpeg -y -i /sdcard/InputFile.mp4 -s 320x240 -r 15 -aspect 3:4 -ab 12288 -vcodec mpeg4 -b 2097152 -sample_fmt s16 /sdcard/out.mp4  


Video Rotate (90 degrees cw):

ffmpeg -y -i /sdcard/videokit/in.mp4 -vf transpose=1 -s 160x120 -r 30 -aspect 4:3 -ab 48000 -ac 2 -ar 22050 -b 2097k /sdcard/video_output/out.mp4


Video Crop:

ffmpeg -y -i /sdcard/videokit/short.mp4 -vf crop=100:100:0:0 -s 320x240 -r 15 -aspect 3:4 -ab 12288 -vcodec mpeg4 -b 2097152 -sample_fmt s16 /sdcard/videokit/out.mp4 


Extract Picture from Video:

ffmpeg -y -i /sdcard/videokit/in.mp4 -an -r 1/2 -ss 00:00:00.000 -t 00:00:03 /sdcard/videokit/filename%03d.jpg


Extract Sound from Video:

ffmpeg -y -i /sdcard/videokit/in.avi -acodec copy /sdcard/videokit/out.mp3


Re-encode Audio in Video:

ffmpeg -y -i /sdcard/in.mp4 -vcodec copy -acodec libmp3lame -ab 64k -ac 2 -b 1200000 -ar 22050 /sdcard/out.mp4


Change Video Resolution:

ffmpeg -y -i /sdcard/in.mp4 -vf transpose=3 -s 320x240 -r 15 -aspect 3:4 -ab 12288 -vcodec mpeg4 -b 2097152 -sample_fmt s16 /sdcard/out.mp4


Cut time segment from Video:

ffmpeg -y -i 101_0077.MP4 -vcodec copy -acodec copy -ss 00:00:00 -t 00:00:01 out.mp4


Transcode Audio:

ffmpeg -y -i /sdcard/videokit/big.wav /sdcard/videokit/small.mp3


Streaming:

// use this command on ffmpeg4android
ffmpeg -i /sdcard/videokit/2.mpg -f mpegts udp://192.168.0.114:8090

// use this command to play the video on the target machine
ffplay -f mpegts -ast 1 -vst 0 -ar 48000 udp://192.168.0.114:8090 

The internals 
As mentioned before, FFmpeg4Android runs as an Android remote service, this service is actually a separate Android process, that encapsulate the FFmpeg native process, that runs directly on the device Linux.
The demo application command will convert vid.mp4 to vid_trans.mp4, it will rotate it 270 degrees, change its resolution to 320x240, change the aspect ratio to 3:4 and encode it with mpeg4 encoder, this is all with one command! (actually it does more, it defines the audio codec, and set the audio bit rate also). 
This is done at the native FFmpeg level, JNI is used to make the calls from the Remote service to the native FFmpeg process.

 Updates - new version - 4.0 
 This use the latest FFmpeg stable source to date, and adds libmp3lame to support mp3 encoding, so now you can easily run commands like this:

* ffmpeg -y -i /sdcard/videokit/in.wav out.mp3
This command transcode WAV file to MP3 file



* ffmpeg -y -i /sdcard/videokit/in.mp4 -vn -ar 44100 -ac 2 -ab 192 -f mp3 /sdcard/videokit/out.mp3

This command extract mp3 sound from the input video. 

Updates - new version - 5.0 
* Version 5 now support Background processing out of the box, no need to write any code for that!

*  Version 5 has streaming support,  to stream use this command: 
String command = "ffmpeg -i /sdcard/videokit/2.mpg -f mpegts udp://192.168.0.114:8090";
To get the streamed video use this on the machine you stream to (in this case 192.168.0.114):
ffplay -f mpegts -ast 1 -vst 0 -ar 48000 udp://192.168.0.114:8090

* Simplified api, all your app has to do is add the library, and with a few lines of code you have FFMpeg support

* Please note that version 5 changed FFMpeg4Android library significantly, hence, most comments below are not relevant.









83 comments:

  1. Hi, i have download ur demo but when i press invoke it does not go in the if statement

    if (remoteService != null) {
    ....
    }
    can u tell me why remoteService variable is empty

    ReplyDelete
  2. You need to start and bind before invoking the service.

    ReplyDelete
  3. yes i have first push the vid.mp4 in my sdcard then i start the app
    then click start then start and then i click invoke

    if (remoteService != null) {
    Toast.makeText(FFMpegClientDemo.this, "Started", Toast.LENGTH_SHORT).show();
    ...
    }

    as u can see have toast in the if and it does not show
    can u plz help i am really stuck... thankx

    ReplyDelete
  4. Yes, first i started and then bind and then i clicked invoke, i have checked in emulator and also my phone.
    HTC Desire android 2.2 but nothing happens when i click invoke i have add toast in the if statement but it will not toast
    plz help

    ReplyDelete
  5. Try to run the demo directly from ffmpeg4android, does it run?

    ReplyDelete
  6. Not clear what u mean by directly form ffmpeg4android\..?

    ReplyDelete
  7. The demo application use ffmeg4android, but its also possible to run the demo from inside ffmpeg4android (the lib also includes the demo app), if its still not clear, read the guide again.

    ReplyDelete
  8. Ok, good now it works, ok the one question i want to use ffmpeg with my app so i downloaded ffmpeg from here
    http://bambuser.com/opensource
    built it and got folder called built/
    so i picked file built/ffmpeg/armeabi-v7a/bin/ffmpeg

    and used this code in my acitvity

    Process process = Runtime.getRuntime().exec("data/data/com.koder.testffmpeg2/files/ffmpeg -version");

    but it gives error
    java.io.IOException: Error running exec(). Command:[/data/data/com.koder.testffmpeg2/files/ffmpeg -version] Working Directory: null Environment:null

    not clear what it mean can u plz help me with this
    i would appriciate it a lot
    Thankx

    ReplyDelete
  9. In order to run it this way, you will need a rooted device.
    Anyway, I'm not familiar with this specific build (there are almost endless possibilities for building ffmpeg, and you need to make sure you chose correctly), anyway, when I build ffmpeg for ffmpeg4android, I had also to make some code changes to make it work on some devices.
    anyway running it via JNI is better, since you will not have to have a rooted phone.

    ReplyDelete
  10. OK, thankx buddy, i dont think i have time to learn ndk thankx anyway

    ReplyDelete
  11. can I use it to stream (using http, rtsp) without converting the video-file. The idea would be to use FFMpeg since it supports much more codecs than native Anroid-code.

    ReplyDelete
    Replies
    1. Yes you can.
      Use this command:
      ffmpeg -i INPUTFILE http://:8090/feed1.ffm
      Where is where ffserver is running, note that if you run ffserver on android, you need to have a rooted device.
      Also, ffserver if not part of ffmgeg4android, You'll need to build and install it separately.
      Note that you can run it on a separate machine.

      Delete
  12. FFmpeg4Android currently does not support streaming.

    ReplyDelete
    Replies
    1. are you planning to support streaming? It would be great - in this way I could show streaming-contents within my app without relying on an external player. I guess I could also use OSD-features (on screen commands). Any plans to implement streaming-support and if yes, any indication when?

      Thanks!

      Delete
    2. This comment has been removed by the author.

      Delete
    3. Yes you can.
      Use this command:
      ffmpeg -i INPUTFILE http://ffserver-host:8090/feed1.ffm
      Where ffserver-host is the host where ffserver is running, note that if you run ffserver on android, you need to have a rooted device.
      Also, ffserver if not part of ffmgeg4android, You'll need to build and install it separately.
      Note that you can run it on a separate machine (and it does not have to be android device).

      Delete
  13. I've been pulling my hair out trying to get FFmpeg to compile with libmp3lame support, while using the Android NDK r7. I was also using the Bambuser build script, and since it sets sysroot to the NDK directory, it can never find libmp3lame. I can build FFmpeg manually, and it finds libmp3lame every time.

    After four days, I've given up.

    I guess my questions is, before I invest time in working with your code:
    1. Can you convert 3GPP/MPEG-4 audio to MP3 (i.e. were you able to add libmp3lame support?)

    ReplyDelete
    Replies
    1. We don't support this external lib, I will add it to our next version wish list.

      Delete
  14. libmp3lame is now supported!

    ReplyDelete
  15. i got remoteService==null when i am invoke then...so can u help to how to reslove this problem...

    ReplyDelete
    Replies
    1. Make sure you start and bind before you invoke.
      Please follow the client demo code.
      If you still have problems, read the Mir posts (above).

      Delete
  16. Hello, My name is Ramiro Coll Doñetz, I am part of the company InfinixSoft. The reason I communicate with you is to consult about your library "FFMPEG 4 Android", which I am interested in implementing one of my applications, which will be distributed and perhaps sell at the Google Play.

    I'm interested to know the price of the license, and if I can use in all applications that I make.

    Also wanted to ask if I can integrate the service into my application, that does not depend on having installed the "FFMPEG 4 Android" from the Google Play.

    Thank you.

    Ramiro Coll Doñetz.

    ReplyDelete
    Replies
    1. Hi Ramiro,
      Please see more pricing and licensing information at:
      http://ffmpeg4android.netcompss.com/
      Regarding your second question, no, you can have it as part of your application.
      If you have more questions,
      Please contact Eli at: eli.hasson@netcompss.com

      Delete
  17. This comment has been removed by the author.

    ReplyDelete
  18. Hi!

    This is a pretty great accomplishment! It is exactly what I needed for my thesis project!

    A question if I may: How would I go about extracting a thumbnail/frame from a
    given video.

    I have tried the usual ffmpeg commands but none work.

    Also as a side note, most of these commands are meant to save the extracted frames to jpg or png files. Is there any logical way to add them to some buffer or bitmap variable?

    I am rather new at ffmpeg so I may be missing some obvious links

    Thank you!

    ReplyDelete
    Replies
    1. Hi Dimitri,
      Here is a command to grab a picture from a video:
      ffmpeg -y -i /sdcard/videokit/in.mp4 -an -r 1/2 -ss 00:00:00.000 -t 00:00:03 /sdcard/videokit/filename%03d.jpg

      Delete
    2. Explanation:
      after the -ss is the time you want to extract the picture from.
      since the rate is 1/2 and the time (-t) is 3 secs, it will grab 6 pictures.

      Delete
  19. Many thanks!

    May I ask one more question?
    What kind of licensing scheme would be needed to use ffmpeg4Android in an academic thesis (published paper)?

    regards!

    ReplyDelete
  20. Please contact Eli from NetComps regarding licensing options:
    eli.hasson@netompss.com

    ReplyDelete
  21. And I'm back with more questions!

    I haven't contacted Eli yet about the license issue but if this next problem is solvable I definitely will (didn't want to bother anyone needlessly)

    So here goes:
    I have managed to get comfortable calling ffmpeg string commands and generating the required thumbnails as image files.
    Now the problem is getting these images into android UI views such as ImageViews etc. as fast as possible.
    What would be the best way to do this? Loading the created files doesn't sound like the best plan, could I somehow send the output straight to an Android bitmap?

    Thanks again for all the help!

    ReplyDelete
    Replies
    1. Hi Dimitry,
      Well, FFMpeg (The native library) is a beast that like to work with files, it does not know what is bitmap, nevertheless Android bitmap, it will take a lot of effort to implement, and I don't think there is a good reason to do so.
      Loading a file from the sdcard, is a very quick operation (few 10 milisecs, depends on the file size), creating a bitmap from the file (that is now loaded into the memory is even quicker), so its actually should work for you very fast,
      Can you test and post here the results?

      Thanks,
      John

      Delete
    2. Hello again!

      I did some tests and indeed loading from the sd card is fast enough!

      The problem now is extracting the frames fast from the video as fast as possible.

      I tried running a single ffmpeg command to extract 1 frame every minute from the video but it is way to slow. For this I used the -r command on the output. Is there maybe a better way?

      The next thing I tried was to extract a single frame with with each command and then seek to the required time position with -ss

      The problem is that apparently I cannot call new ffmpeg commands in rapid succession.

      I am using a slightly modified version of the Invoke function from the demo. In LogCat a serviceDisconnected message appears after every invoke so I suspect that I must find a way not to disconnect the service after every call (if indeed that is what is happening)

      Any insight? I do apologise for the barrage of questions

      cheers!

      Delete
  22. Updated client source to the latest version.

    Eli

    ReplyDelete
  23. Does the current version support streaming from a usb mic or usb camera? What streaming protocols are supported? Can i stream from a usb mic to say a flash or wowza server?. During a streaming process can i transcode from mpeg2 to mpeg4?

    ReplyDelete
    Replies
    1. I would like to know this also, I have tried this with no go:

      ffmpeg -i /dev/graphics/fb0 http://ip:8090/cam1.ffm

      I have also tried:

      ffmpeg -i /dev/video0 http://ip:8090/cam1.ffm

      Neither works, no output from FFMpeg.

      Delete
  24. Is it possible to redirect ffmpeg to accept OutputStream as its input for encoding?

    ReplyDelete
  25. Its not tested, its possible it will work.

    ReplyDelete
  26. Hi.
    Is now the "FFMpeg4Android project library" free? I saw a `if (! isLicenseValid()) return;`.
    Thanks.

    ReplyDelete
  27. Its free for personal use,
    If you want to bundle it with your app, the price is 399USD

    ReplyDelete
  28. Intent intent = new Intent();
    intent.setClassName("com.netcompss.ffmpeg4android", "FFMpegRemoteServiceBridge");
    ComponentName cn = startService(intent);

    startService(intent) returns null!

    ReplyDelete
    Replies
    1. Is there any clear guide on how to start and bind to the service?

      Thanks.

      Delete
    2. Lets start from the end, does the apk works for you?
      If yes, can you compile and run the lib on your device?

      Delete
  29. Hi Eli,

    Sorry I don't get what you mean by "the apk".

    The Demo app works on my device. I downloaded the sources and put the service in my build path in Eclipse. It is built without error. But I cannot get the service running.

    ReplyDelete
    Replies
    1. I have also declared it in the Android manifest.

      Delete
    2. The compiled demo client does not run on the device:

      Unable to resolve supperclass of Lcom/examples/ffmpegforAndroid_demo/DemoClient;

      Delete
    3. you need to install ffmpeg4android apk (from the market) or compile and run it via eclipse before you run the demo.

      Delete
    4. Well that's the very first thing I did before going for the source codes! It works very well on my device and I tested several commands on it.

      Unless there is a compatibility issue between the source that I downloaded and the version that I have on my device (SGS3).

      Delete
    5. I just downloaded the latest version (5.0.2) of ffmpeg4android and installed it. The Demo client cannot link to the class and crashes.

      Please advise.

      Delete
    6. There is no startService call in the demo client?

      Delete
    7. No, version 5 is much more simple, basically you set the command and call run, the framework does everything else.
      Looks like Your problem relates to the version change.,
      You need to make sure that everything (apk, ffmpeg4android, and demo client) are 5.0.2

      Delete
    8. All are 5.0.2, the apk works fine, but compiled Demo client crashes when "invoke" button is clicked, my own application also crashes when invokeService() is called:

      at com.netcompss.ffmpeg4android_client.BaseWizard.is LicenseValid(BaseWizard.java:100)

      Delete
    9. Try this (it worked for me on a fresh machine):
      1. download, import and compile the 2 projects (ffmpeg4android, client demo).
      Verify that in both the version is 5.0.2

      2. Go to ffmpeg4android properties, unchecked the "is library" checkbox, and click the run button (with your device connected), this will install ffmpeg4android apk on your device.

      3. Run the demo client (with your device connected), try to run the command, it should work now.

      Delete
    10. I found the issue, it seems that for some reason the blog post was not updated with the correct link to version 5.0.2, so you downloaded an older version.
      I fixed it, now both the links are directing to the correct location (version 5.0.2 for the demo client, and the ffmpeg4android library)

      Delete
    11. Yes. Doing this it works.
      But is it supposed to work also if we launch the DemoClient project with ffmpeg4android checked as library?

      Delete
    12. No, the demo client must have the apk installed.
      If you want it to work without the apk, you should bundle your code with the ffmpeg4andoid project, where com.example package is the place holder for your code.

      Delete
    13. @John Here is what I tried:

      1- Downloaded everything version 5.0.2 for sure 100%
      2- Imported Demo Client project into Eclipse
      3- Imported FFmpeg4Adnroid into Eclipse
      4- Uninstalled everything on my device (previous Demo and ffmpeg4Adroid)
      5- Run FFmpeg4Android as you said, to have the apk on my device
      6- Run Demo Client, it freezes after I click invoke
      7- Run my application, invokeService() is called but an exception occurs as below (in Logcat):

      dalvikvm Exception Ljava/lang/UnsatisfiedLinkError; thrown while initializing Lcom/netcompss/ffmpeg4android/LicensecheckJNI;

      then:

      dalvikvm threadid=1: thread exiting with uncaught exception (group=0x....)

      AndroidRuntime FATAL EXCEPTION: main
      .
      .
      .
      and lots of error messages regarding LicenseCheckJNI (and this one: Couldn't load license-jni: findLibrary returned null)

      Delete
  30. the apk from the market?
    Anyway, perfectly clear.

    ReplyDelete
  31. Hi guys,
    Your package looks good!
    What happens if I use ffplay -i rtp://@224.0.0.1:1234 in the demo (Multicast RTP)
    Should it work?
    Phil

    ReplyDelete
    Replies
    1. This was verifed to work:
      ffplay -f mpegts -ast 1 -vst 0 -ar 48000 udp://192.168.0.114:8090
      String command = "ffmpeg -i /sdcard/videokit/2.mpg -f mpegts udp://192.168.0.114:8090";

      Regarding RTP, I was able to send RTP packages using this command:
      ffmpeg -i in.mp4 -an -f rtp rtp://192.168.0.109:20000
      I sniffed the packages on the target machine, But I was not able to play them.

      Delete
    2. Just to clarify,
      The: ffplay -f mpegts -ast 1 -vst 0 -ar 48000 udp://192.168.0.114:8090
      Is run on the target machine to play the streamed video, when ffmpeg4android used this command to stream:
      String command = "ffmpeg -i /sdcard/videokit/2.mpg -f mpegts udp://192.168.0.114:8090";

      Delete
    3. thanks John, can you confirm that the "streamer" side was run on a different machine than the one where you are playing the stream (192.168.0.114)?

      Delete
  32. @John just repeated the above post in case you haven't seen this.

    Here is what I tried:

    1- Downloaded everything version 5.0.2 for sure 100%
    2- Imported Demo Client project into Eclipse
    3- Imported FFmpeg4Adnroid into Eclipse
    4- Uninstalled everything on my device (previous Demo and ffmpeg4Adroid)
    5- Run FFmpeg4Android as you said, to have the apk on my device
    6- Run Demo Client, it freezes after I click invoke
    7- Run my application, invokeService() is called but an exception occurs as below (in Logcat):

    dalvikvm Exception Ljava/lang/UnsatisfiedLinkError; thrown while initializing Lcom/netcompss/ffmpeg4android/LicensecheckJNI;

    then:

    dalvikvm threadid=1: thread exiting with uncaught exception (group=0x....)

    AndroidRuntime FATAL EXCEPTION: main
    .
    .
    .
    and lots of error messages regarding LicenseCheckJNI (and this one: Couldn't load license-jni: findLibrary returned null)

    It seems LicenseCheckJNI is trying to create resources that are not accessible when it is bundled with my application. I think I cannot use it in my app...I am going to build ffmpeg under Ubuntu and use it in my app instead.

    ReplyDelete
  33. You can remove this call:
    if (! isLicenseValid()) return;
    from the DemoClient.java

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
    2. Alex,
      Sorry for the lack of docs, till then, I'm here to answer any question.
      The ffmpeg4android source is available for use and change.
      Note that in ffmpeg4android project, com.example is a place holder for your code, this is the quickest way to bundle your app with ffmpeg4android.
      Of-course, you will need to refactor the project name, add your resources, etc,
      but this is relatively very easy.

      You can also add the code to your project, but in this case you need to be more careful, and make sure you have everything (packages, resources), and do it slowly and gradually.

      I wish it could work as a library, the problem is that it looks like ADT does not support NDK results in a library.

      Delete
  34. Hi, i am trying to extract info of the re-encoded video i just created and i entered this command
    ffmpeg -i /sdcard/videokit/A16.mp4
    but i got this error "At least one output file must be specified" and i assumed i needed a text file so i edited the command to
    ffmpeg -i /sdcard/videokit/A16.mp4 /sdcard/videokit/out.txt
    but i got an invalid argument error.
    How can i extract the video information. I have looked at the log but i require more details and some information in the log is contradicting, for instance there are two bitrates or maybe i am the one who doesn't know how to interpret the log. Please let me know how i can extract the video information. Thanx

    ReplyDelete
    Replies
    1. Your first command is fine, the information will show in the log file in the videokit folder, and also you can view it from the app using the show last run log (in the options menu).

      Delete
  35. This comment has been removed by the author.

    ReplyDelete
  36. I want to change bitrate of MP3 file from 320kb/s to 80kb/s, In FFmpeg 4 Android I typed this command:
    ffmpeg -i /sdcard/320.mp3 -ab 80k /sdcard/80.mp3
    but it not success, can you tell me why ?

    ReplyDelete
  37. Here is a working command:

    ffmpeg -y -i /sdcard/videokit/in.mp3 -ab 128k -ac 2 -b 1200000 -ar 22050 /sdcard/videokit/out.mp3

    ReplyDelete
  38. Thank you so much, it work good, but I have a problem. Now I want to build an Android simple project, such as:
    Input: a mp3 file with (128kb/s)
    Ouput : a mp3 file with (80kb/s)
    When I click on "Convert" button, it will start transfering process. Could you tell me what steps must I follow to do that.

    ReplyDelete
  39. I'm trying to get this implemented into my existing project, and I'm running into some issues:

    Where is this vk.log file supposed to be created? Am I missing something about the service that should be happening in the background? I keep getting LogCat errors saying that the getDutationFromVCLogRandomAccess() method has failed:

    05-09 15:14:49.851: I/ffmpeg4android(8678): ========calc progress=======
    05-09 15:14:49.859: W/ffmpeg4android(8678): /storage/sdcard0/videokit/vk.log: open failed: ENOENT (No such file or directory)
    05-09 15:14:49.859: W/ffmpeg4android(8678): Prefs.durationOfCurrent is null
    05-09 15:14:49.859: I/ffmpeg4android(8678): onProgressUpdate: 0
    05-09 15:14:50.132: I/ffmpeg4android(8678): Cancel clicked
    05-09 15:14:50.132: I/ffmpeg4android(8678): Setting Prefs.forceStopFlag to true
    05-09 15:14:50.140: I/ffmpeg4android(8678): DeadObject Exception after fexit()

    Any ideas?

    ReplyDelete
    Replies
    1. Is this an issue with the ffmpeg service wanting to put the output in the /sdcard/videokit/ folder instead of what I actually have which is /storage/sdcard0/videokit?

      Delete
    2. I found my error, FFMPEG doesn't like spaces in the name.

      Delete
  40. Hi,
    Is this library supports Video4linux2 (v4l2) as source of video stream (/dev/video*) and will it be possible to stream video to Wowza Media Servers ?

    With thanks and regards

    Prasad Munaga

    ReplyDelete
    Replies
    1. Sorry, its not supported.

      Delete
    2. Hi John,

      Is there any other solution to achieve the desired result as my project got stuck over on this issue. Your kind help is very much appreciated.

      With thanks and regards

      Prasad Munaga

      Delete