Media3 1.5.0 — what’s new?

Posted by Kristina Simakova – Engineering Supervisor

This text is cross-published on Medium

Transformer now helps movement pictures and quicker picture encoding. We’ve additionally simplified the setup for DefaultPreloadManager and ExoPlayer, making it simpler to make use of. However that’s not all! We’ve included a brand new IAMF decoder, a Kotlin listener extension, and simpler Participant optimization via delegation.

To be taught extra about all new APIs and bug fixes, try the full launch notes.

Transformer enhancements

Movement photograph help

Transformer now helps exporting movement pictures. The movement photograph’s picture is exported if the corresponding MediaItem’s picture length is ready (see MediaItem.Builder().setImageDurationMs()) In any other case, the movement photograph’s video is exported. Notice that the EditedMediaItem’s length shouldn’t be set in both case as it’s going to routinely be set to the corresponding MediaItem’s picture length.

Sooner picture encoding

This launch accelerates image-to-video encoding, due to optimizations in DefaultVideoFrameProcessor.queueInputBitmap(). DefaultVideoFrameProcessor now treats the Bitmap given to queueInputBitmap() as immutable. The GL pipeline will resample and color-convert the enter Bitmap solely as soon as. Because of this, Transformer operations that take massive (e.g. 12 megapixels) pictures as enter execute quicker.

AudioEncoderSettings

Just like VideoEncoderSettings, Transformer now helps AudioEncoderSettings which can be utilized to set the specified encoding profile and bitrate.

Edit listing help

Transformer now shifts the primary video body to start out from 0. This fixes A/V sync points in some recordsdata the place an edit listing is current.

Unsupported observe sort logging

This launch contains improved logging for unsupported observe sorts, offering extra detailed data for troubleshooting and debugging.

Media3 muxer

In one of many earlier releases we added a brand new muxer library which can be utilized to create MP4 container recordsdata. The media3 muxer affords help for a variety of audio and video codecs, enabling seamless dealing with of various media codecs. This new library additionally brings superior options together with:

The muxer library will be included as a gradle dependency:

implementation ("androidx.media3:media3-muxer:1.5.0")

Media3 muxer with Transformer

To make use of the media3 muxer with Transformer, set an InAppMuxer.Manufacturing facility (which internally wraps media3 muxer) because the muxer manufacturing unit when making a Transformer:

val transformer = Transformer.Builder(context)
    .setMuxerFactory(InAppMuxer.Manufacturing facility.Builder().construct())
    .construct()

Easier setup for DefaultPreloadManager and ExoPlayer

With Media3 1.5.0, we added DefaultPreloadManager.Builder, which makes it a lot simpler to construct the preload elements and the participant. Beforehand we requested you to instantiate a number of required elements (RenderersFactory, TrackSelectorFactory, LoadControl, BandwidthMeter and preload / playback Looper) first, and be tremendous cautious on appropriately sharing these elements when injecting them into the DefaultPreloadManager constructor and the ExoPlayer.Builder. With the brand new DefaultPreloadManager.Builder this turns into so much easier:

• Construct a DefaultPreloadManager and ExoPlayer situations with all default elements.

val preloadManagerBuilder = DefaultPreloadManager.Builder()
val preloadManager = preloadManagerBuilder.construct()
val participant = preloadManagerBuilder.buildExoPlayer()

• Construct a DefaultPreloadManager and ExoPlayer situations with customized sharing elements.

val preloadManagerBuilder = DefaultPreloadManager.Builder().setRenderersFactory(customRenderersFactory)
// The ensuing preloadManager makes use of customRenderersFactory
val preloadManager = preloadManagerBuilder.construct()
// The ensuing participant makes use of customRenderersFactory
val participant = preloadManagerBuilder.buildExoPlayer()

• Construct a DefaultPreloadManager and ExoPlayer situations, whereas setting the customized playback-only configurations on the ExoPlayers.

val preloadManagerBuilder = DefaultPreloadManager.Builder()
val preloadManager = preloadManagerBuilder.construct()
// Tune the playback-only configurations
val playerBuilder = ExoPlayer.Builder().setFooEnabled()
// The ensuing participant can have playback function "Foo" enabled
val participant = preloadManagerBuilder.buildExoPlayer(playerBuilder)

Preloading the subsequent playlist merchandise

We’ve added the flexibility to preload the subsequent merchandise within the playlist of ExoPlayer. By default, playlist preloading is disabled however will be enabled by setting the length which ought to be preloaded to reminiscence:

participant.preloadConfiguration =
    PreloadConfiguration(/* targetPreloadDurationUs= */ 5_000_000L)

With the PreloadConfiguration above, the participant tries to preload 5 seconds of media for the subsequent merchandise within the playlist. Preloading is barely began when no media is being loaded that’s required for the continued playback. This fashion preloading doesn’t compete for bandwidth with the first playback.

When enabled, preloading may help decrease be a part of latency when a person skips to the subsequent merchandise earlier than the playback buffer reaches the subsequent merchandise. The primary interval of the subsequent window is ready and video, audio and textual content samples are preloaded into its pattern queues. The preloaded interval is later queued into the participant with preloaded samples instantly obtainable and able to be fed to the codec for rendering.

As soon as opted-in, playlist preloading will be turned off once more through the use of PreloadConfiguration.DEFAULT to disable playlist preloading:

participant.preloadConfiguration = PreloadConfiguration.DEFAULT

New IAMF decoder and Kotlin listener extension

The 1.5.0 launch features a new media3-decoder-iamf module, which permits playback of IAMF immersive audio tracks in MP4 recordsdata. Apps wanting to do this out might want to construct the libiamf decoder domestically. See the media3 README for full directions.

implementation ("androidx.media3:media3-decoder-iamf:1.5.0")

This launch additionally features a new media3-common-ktx module, a house for Kotlin-specific performance. The primary model of this module accommodates a droop perform that lets the caller take heed to Participant.Listener.onEvents. It is a constructing block that’s utilized by the upcoming media3-ui-compose module (launching with media3 1.6.0) to energy a Jetpack Compose playback UI.

implementation ("androidx.media3:media3-common-ktx:1.5.0")

Simpler Participant customization by way of delegation

Media3 has supplied a ForwardingPlayer implementation since model 1.0.0, and we now have beforehand steered that apps ought to use it once they need to customise the way in which sure Participant operations work, through the use of the decorator sample. One quite common use-case is to permit or disallow sure participant instructions (with a view to present/conceal sure buttons in a UI). Sadly, doing this appropriately with ForwardingPlayer is surprisingly laborious and error-prone, as a result of it’s important to constantly override a number of strategies, and deal with the listener as properly. The instance code to reveal how fiddly that is too lengthy for this weblog, so we’ve put it in a gist as an alternative.

With the intention to make these types of customizations simpler, 1.5.0 features a new ForwardingSimpleBasePlayer, which builds on the consistency ensures supplied by SimpleBasePlayer to make it simpler to create constant Participant implementations following the decorator sample. The identical command-modifying Participant is now a lot easier to implement:

class PlayerWithoutSeekToNext(participant: Participant) : ForwardingSimpleBasePlayer(participant) {
  override enjoyable getState(): State {
    val state = tremendous.getState()
    return state
      .buildUpon()
      .setAvailableCommands(
        state.availableCommands.buildUpon().take away(COMMAND_SEEK_TO_NEXT).construct()
      )
      .construct()
  }

  // We needn't override handleSeek, as a result of it's assured to not be referred to as for
  // COMMAND_SEEK_TO_NEXT since we have marked that command unavailable.
}

MediaSession: Command button for media gadgets

Command buttons for media gadgets enable a session app to declare instructions supported by sure media gadgets that then will be conveniently displayed and executed by a MediaController or MediaBrowser:

You may discover the detailed documentation on android.developer.com.

That is the Media3 equal of the legacy “customized browse actions” API, with which Media3 is totally interoperable. Not like the legacy API, command buttons for media gadgets don’t require a MediaLibraryService however are a function of the Media3 MediaSession as an alternative. Therefore they’re obtainable for MediaController and MediaBrowser in the identical means.

In the event you encounter any points, have function requests, or need to share suggestions, please tell us utilizing the Media3 difficulty tracker on GitHub. We look ahead to listening to from you!