This commit un-deprecates DeprecatedString, and repurposes it as a byte
string.
As the null state has already been removed, there are no other
particularly hairy blockers in repurposing this type as a byte string
(what it _really_ is).
This commit is auto-generated:
$ xs=$(ack -l \bDeprecatedString\b\|deprecated_string AK Userland \
Meta Ports Ladybird Tests Kernel)
$ perl -pie 's/\bDeprecatedString\b/ByteString/g;
s/deprecated_string/byte_string/g' $xs
$ clang-format --style=file -i \
$(git diff --name-only | grep \.cpp\|\.h)
$ gn format $(git ls-files '*.gn' '*.gni')
Since it will become a stream in a little bit, it should behave like all
non-trivial stream classes, who are not primarily intended to have
shared ownership to make closing behavior more predictable. Across all
uses of MappedFile, there is only one use case of shared mapped files in
LibVideo, which now uses the thin SharedMappedFile wrapper.
`seek_demuxer_to_most_recent_keyframe()` wasn't correctly returning in
cases where an error was thrown by the demuxer. To avoid this, the
function now returns the error, and the playback state handler must act
on it instead, allowing it to exit the seeking state early.
Previously, we would unwrap the duration value even when it contained
an error, causing an assertion failure.
Later, we should change it to return the last known sample timestamp
in the media data, allowing for example live-streamed video to have
a semi-useful duration. In general, though, this is not used by
players in the wild, so we can leave it for now.
That's what this class really is; in fact that's what the first line of
the comment says it is.
This commit does not rename the main files, since those will contain
other time-related classes in a little bit.
In addition, this renames the internal timer to better suit its new
purpose since the playback state handlers were added. Not only is it
used to time frame presentations, but also to poll the queue when
seeking or buffering.
Running decoding on the main thread can cause frames to be delayed due
to the presentation timer waiting for a decode to finish. If a frame
takes too long to decode, this delay can be quite visible. Therefore,
decoding has been moved to a separate thread, with frames sent to the
main thread through a thread-safe queue.
This results in frame times going from being late by up to 16ms to a
very consistent ~1-2ms.
This allows us to create a PlaybackManager from a file which has already
been mapped, instead of passing a file name.
This means that anyone who uses `PlaybackManager` can now use LibFSAC :)
Previously, there was some leftover logic in `SeekingStateHandler`
which would avoid presenting a frame if it didn't find a frame both
before and after the target frame. However, this logic was unnecessary
as the `on_enter()` function would check if it needed to present a
frame and exit seeking if not.
This allows seeking to succeed if the Seeking handler cannot find a
frame before the one to be seeked to, which could occur if the first
frame of a video is not at timestamp 0.
This adds a timestamp to the debug output when presenting a frame, so
it can be clear the frame spacing between a presented frame and a state
change.
The seeking state will also now print when it early-exits if the seek
point is within the current sample's duration.
StartingStateHandler and SeekingStateHandler were declaring their own
`bool m_playing` fields (from previous code where there was no base
class).
In the case of SeekingStateHandler, this only made the logging wrong.
For StartingStateHandler, however, this meant that it was not using
the boolean passed as a parameter to the constructor to define the
state that would be transitioned to after the Starting state finished.
This meant that when the Stopping state replaced itself with the
Starting state, playback would not resume when Starting state exits.
To pass events from LibVideo's PlaybackManager to interested parties, we
currently dispatch Core::Event objects that outside callers listen for.
Dispatching events in this manner rely on a Core::EventLoop. In order to
use PlaybackManager from LibWeb, change this mechanism to instead use a
set of callbacks to inform callers of events.
Using uninitialized primitives is undefined behavior. In this case, all
videos would autoplay on Serenity, but not play at all on Lagom, due to
some m_playing booleans being uninitialized.
This allows the logic for keeping track of whether to resume to the
paused or the playing state when exiting these states. The new
StartingStateHandler also uses the class, since it can also be paused
and unpaused while waiting for samples.
The pause/play actions on the handlers inheriting from the resuming
handler will also now notify the owner that the state has changed so
that it can change icons, etc.
The PlaybackStateChangeEvent wasn't connected up anymore, so the player
wouldn't change icons when stopping playback due to reaching the end of
the stream or encountering an error.
This new state handler will retrieve and display the first frame, while
ensuring that playback can start as soon as possible by buffering two
frames on top of the first frame for the PlayingStateHandler to set its
next frame timer by.
Previously, we assumed the timer was hitting at the correct time. This
meant that if we changed states and the previous state handler had
prepared a next frame, we would immediately display it without checking
the timestamp.
If we encounter an error in the video decoder, we can set a timestamp
for the error item in the queue so that it will display the error only
when the frame that caused the error would have been displayed.
Previously we had dispatch_decoder_error and on_decoder_error serving
the same function, with one not handling the end of stream properly.
There is also a new function to dispatch either an error or a frame to
the owner of this playback manager, so that PlaybackStateHandlers don't
have to duplicate this logic.
For example, consider cases where we want to propagate errors only in
specific instances:
auto result = read_data(); // something like ErrorOr<ByteBuffer>
if (result.is_error() && result.error().code() != EINTR)
continue;
auto bytes = TRY(result);
The TRY invocation will currently copy the byte buffer when the
expression (in this case, just a local variable) is stored into
_temporary_result.
This patch binds the expression to a reference to prevent such copies.
In less trival invocations (such as TRY(some_function()), this will
incur only temporary lifetime extensions, i.e. no functional change.
That matches the terminology used in ITU-T Rec. H.273,
PNG's cICP chunk, and the ICC cicpTag.
Also change the enum values to match the values in the spec --
0 means "not full range" and 1 means "full range".
(For now, keep the "Unspecified" entry around, and give it value 2.
This value is not in the spec.)
No intended behavior change.
Fast seeking does not work correctly when seeking in small increments,
so it is necessary to use accurate seeking when using certain actions.
The PlaybackManager has been changed to accept the seek mode as a
parameter to `seek_to_timestamp` to facilitate this. This now means
that it no longer has to track a seek mode preference.
In cases where the PlaybackManager's earliest buffered or displayed
sample is closer to the seek target than the demuxer's chosen keyframe,
we don't want to seek at all. To enable this, demuxers now receive an
optional parameter with the earliest timestamp that the caller can
still access.
The demuxer in turn returns an optional to indicate when a seek was not
needed, which allows PlaybackManager to avoid clearing its queue and
re-decoding frames.
Storing playback states in virtual classes allows the behavior to be
much more clearly written. Each `PlaybackStateHandler` subclass can
implement some event-handling functions to model their behavior, and
has functions to change its parent PlaybackManager's state to any other
state.
This will allow expanding the functionality of playback in the future,
for example to allow skipping a single frame forward/backward.
A bit of a bikeshed, but status sounds more like a result of an action,
and state sounds more accurate to what the `PlaybackManager` does.
The previous and current state fields of the `PlaybackStateChangeEvent`
are now removed, since they were unused (for now).
This is a lead-up to the refactoring of VideoPlaybackManager to make
that diff more legible.
If USING_AK_GLOBALLY is not defined, the name IsLvalueReference might
not be available in the global namespace. Follow the pattern established
in LibTest to fully qualify AK types in macros to avoid this problem.
This will make it easier to support both string types at the same time
while we convert code, and tracking down remaining uses.
One big exception is Value::to_string() in LibJS, where the name is
dictated by the ToString AO.
When errors are encountered by PlaybackManager, it attempts to switch
states to either Stopped or Corrupted. However, that causes it to set
the last presentation media time to the current playback time while the
last presentation time is unexpectedly negative because the seek never
ended.
Ending the seek before the state changes to Stopped or Corrupted
prevents this situation from happening.
This implements the fastest seeking mode available for tracks with cues
using an array of cue points for each track. It approximates the index
based on the seeking timestamp and then finds the earliest cue point
before the timestamp. The approximation assumes that cues will be on
a regular interval, which I don't believe is always the case, but it
should at least be faster than iterating the whole set of cue points
each time.
Cues are stored per track, but most videos will only have cue points
for the video track(s) that are present. For now, this assumes that it
should only seek based on the cue points for the selected track. To
seek audio in a video file, we should copy the seeked iterator over to
the audio track's iterator after seeking is complete. The iterator will
then skip to the next audio block.
Now that we're able to find the nearest keyframe, we can have a fast
seeking mode that only seeks to keyframes, so that it doesn't have to
also decode inter frames until it reaches the timestamp.
The default is still accurate seeking, so that the entire seeking
implementation can be tested.
This implements the PlaybackManager portion of seeking, so that it can
seek to any frame in the video by dropping all preceding frames until
it reaches the seek point.
MatroskaDemuxer currently will only seek to the start of the video.
That means that every seek has to drop all the frames until it comes
across the target timestamp.
The PlaybackManager::update_presented_frame function was getting out of
hand and adding seeking was making it illegible. This rewrites it to be
(hopefully) quite a bit more readable, and adds a few comments to help
future readers of the code.
In addition, some helpful debugging prints were added that should help
debug any future issues with the player.
With these changes, the seek bar can be used, but only to seek to the
start of the file. Seeking to anywhere else in the file will cause an
error in the demuxer.
The timestamp label that was previously invisible now has its text set
according to either the playback or seek slider's position.
The Demuxer class was changed to return errors for more functions so
that all of the underlying reading can be done lazily. Other than that,
the demuxer interface is unchanged, and only the underlying reader was
modified.
The MatroskaDocument class is no more, and MatroskaReader's getter
functions replace it. Every MatroskaReader getter beyond the Segment
element's position is parsed lazily from the file as needed. This means
that all getter functions can return DecoderErrors which must be
handled by callers.
