The monitoring deduplication was previously checking at each audio tick
the whole audio tree for Audio Output Capture (AOC) devices used for
monitoring. The profiling did not show any big impact on the audio
callback. But due to reports of a significant slowdown for scenes with
numerous audio sources, we have moved the check on AOC from the audio
thread to the UI thread.
So we implemented obs_source_audio_output_capture_device_changed which
is triggered whenever:
- a monitoring device is changed in Settings > Audio,
- an Audio Output Capture source changes its device or on startup.
This function compares the AOC device with the monitoring device in UI
thread. If they are identical, a signal is sent to the audio thread to
add an audio task updating the AOC duplicating source pointer at the end
of an audio tick.
This triggers monitoring deduplication if the ptr is not NULL.
The calls in the AOC are implemented in next commits.
The rest of the logic in obs-audio.c is the same except on one count,
which is that we check against the muted state of the AOC rather than
its user_muted; with the new logic, muted works better.
Signed-off-by: pkv <pkv@obsproject.com>
This adds comparison to default devices to the monitoring deduplication.
When a user picks a default device, the device_id setting is 'default',
which prevents any comparison.
The comparison is done by leveraging the libobs/audio-monitoring
devices_match function.
For macOS, some special care is taken because the devices list differ
for 'Desktop Audio' and 'monitoring' since coreaudio sdk has no pure
audio capture; so 'default' in the two lists do not match in general.
One then retrieves the device_id for the default desktop audio for macOS
through get_desktop_default_id function.
Signed-off-by: pkv <pkv@obsproject.com>
When the 'Audio Output Capture' source (usually Desktop Audio) has its
fader at minimum, we should disable the deduplication logic.
This is done by checking against the obs_source volume member.
Signed-off-by: pkv <pkv@obsproject.com>
Monitoring deduplication must be applied only to tracks for which the
monitored source and the 'Audio Output Capture' source are both enabled.
This adds such checks.
Signed-off-by: pkv <pkv@obsproject.com>
The commit adds a log line to inform the user that deduplication is
being applied. The info is displayed whenever deduplication first
occurs.
Signed-off-by: pkv <pkv@obsproject.com>
When an 'Audio Capture Source' device is also used for monitoring, the
deduplication logic is applied: all monitored sources are silenced.
But this should not silence the 'Audio Capture Source' if for some
reason, it is being monitored (the user will get echoes, but hey, it's
their choice). So we exclude the 'Audio Capture Source' from the
silenced sources.
Signed-off-by: pkv <pkv@obsproject.com>
This fixes the following issue:
- when an Audio Capture source (like 'Desktop Audio',
'wasapi_output_capture' or 'pulse_output_capture') and monitoring use
the same device, one gets duplicated audio for any monitored source,
since their audio is added through both the current scene and the output
capture.
This creates a shift in audio level (about 3 dBFS for white noise, and
oftentimes for music; up to 6 dBFS for very low frequency sine waves).
We solve the bug by:
- detecting when an Audio Capture source and monitoring share the same
device;
- silencing monitored sources whenever the Audio Capture source is
unmuted.
Signed-off-by: pkv <pkv@obsproject.com>
This fixes the following bug:
- a source might be copied into the same scene or through a nested scene.
The audio level will then increase by +6 dBFS.
An earlier fix [1] dealt with this bug at the scene audio rendering
level, which leaves some edge cases since the fix is not located
directly in the core audio callback.
The current fix consists in:
- tagging individual sources which appear several times in the audio
tree at each tick;
- promote them to root_nodes sources;
- bypass their mixing in scenes and transitions.
Due to being mixed as root_nodes, the audio of duplicated sources
appears only once in the final audio mix.
[1] https://github.com/obsproject/obs-studio/pull/10537
Signed-off-by: pkv <pkv@obsproject.com>
This adds a new obs_canvas object that acts as a shareable
(reference-counted) owner of views, mixes, and (optionally) scenes.
This is a step towards faciliatating multi-canvas and multi-output
features in OBS Studio.
It solves a number of complications that exist with the manual approach
of using views, such as audio mixing, source active-state tracking, and
scenes not havinga reliable way of identifying the actual available
canvas size.
Allows a frontend the ability to set the maximum audio buffering
latency, and specify whether that audio buffering is either fixed (to
the maximum audio buffering latency), or dynamically increasing from 0.
This will be useful if the user wishes to output audio to devices or
through a virtual audio device at a guaranteed minimal latency.
When audio buffering is maxed, certain sources will always repeat the
same debug logging message of "render audio source [name] has gone
backwards", which shouldn't apply if the audio timestamp is 0 rather
than a valid audio value.
This is in anticipation to adding low audio latency mode, will cause
audio buffering to be fixed rather than dynamically increasing. Having
fixed audio buffering means that audio latency always stays the same
rather than dynamically increasing.
df4eb82 fixed a bug that caused source audio timestamps to perpetually
lag. However, there is a deeper issue where after we reach max
buffering, lagging sources make OBS's entire audio pipeline fall over.
These may be corrected by later code, but still cause global audio
glitches at best. Persistent problems, as prior to df4eb82, cause audio
to fail entirely.
The root cause is that OBS's audio mixing tree cannot deal with
timestamps prior to the current audio tick. Intermediate mixing stages
assume that the lowest incoming timestamp is the base of the current
tick, and mix accordingly. This propagates lagged timestamps up the
tree, where at the top level mix_audio will drop the source entirely -
which at this point is a transition covering all inputs, thus glitching
audio globally. Where extra buffering can cover the slip, the entire mix
gets retried and the error corrected, but when the global buffer
duration is maxed out, it makes it to the output.
The solution is to catch laggy sources immediately after rendering, and
drop audio to bring them back in sync, or mark them pending if not
enough audio is available. This ensures later mixing stages are not fed
with out of sync timestamps.
This improves the ignore_audio code to only drop as much audio as
needed to bring the source back in sync, and moves its call to
immediately after source audio rendering.
Regression introduced by dc4e20500: while the stop detection is pending,
it should still return false so the rest of the discard code can run.
Otherwise, the source audio will remain in the buffer, lagging the
source and triggering audio buffering increases until max audio
buffering is reached.
As os_gettime_ns() gets large the current scaling methods, mostly by casting
to uint64_t, may lead to numerical overflows. Sweep the code and use
util_mul_div64() where applicable.
Signed-off-by: Hans Petter Selasky <hps@selasky.org>
If an audio source does not provide enough data at a steady pace, the
timestamp update does not happen, and buffering increases until it
maxes out. To counteract this, update the timestamp anyway.
Another issue for decoupled audio sources is that timing is not
adjusted for divergence from system time. Making this adjustment is
better for timing stability.
5+ hours of stable audio without any buffering on my GV-USB2 where it
used to add 21ms every 5 mintues or so.
Fixes https://obsproject.com/mantis/view.php?id=1269
Code submissions have continually suffered from formatting
inconsistencies that constantly have to be addressed. Using
clang-format simplifies this by making code formatting more consistent,
and allows automation of the code formatting so that maintainers can
focus more on the code itself instead of code formatting.
Uses obs_source_get_ref on the sources enumerated in the tick_sources
function in obs-video.c to ensure a reference has been incremented
before calling that source's video_tick, and replaces an
obs_source_addref with obs_source_get_ref in the push_audio_tree
function in obs-audio.c to ensure that it cannot increment a source that
has already decremented its reference to 0.
The default buffering time for audio was always 1 second before the
audio subsystem was changed, and it was always more than sufficient for
max audio buffering time
This code causes audio data in general to be reset (and subsequently
deleted). It should just be marked as pending and ignored until the
data is ready. The discard_if_stopped function will serve the same
purpose if the source's audio has actually stopped.
This variable is used to detect whether audio has stopped -- if audio
stops, it detects that no new data is coming in, and resets the audio
position so that it eliminates the chance of causing the audio buffering
to go haywire if audio starts up again. However, this variable was not
being reset every time the value changes, which it should.
If the circular audio buffer of the source has data remaining that's
less than the audio frame tick count (1024 frames), it would just leave
that audio data on the source without discarding it. However, this
could cause audio buffering to increase unnecessarily under certain
circumstances (when the next audio timestamp is within the timestamp
jump window), so it would append data to that circular buffer despite
the audio stopping that long ago, causing audio buffering to have to
increase to compensate.
Instead, just discard pending audio if it hasn't been written to. In
other words, if the audio has stopped and there's insufficient audio
left to continue processing.
Fixes an issue where audio data would not be popped if they were not
activated/presenting. This would cause the audio subsystem to
needlessly buffer when they were reactivated again. Rendering all audio
sources (excuding composite/filter sources) helps ensure that audio data
is always popped and not left to pile up.
The new audio subsystem fixes two issues:
- First Primary issue it fixes is the ability for parent sources to
intercept the audio of child sources, and do custom processing on
them. The main reason for this was the ability to do custom
cross-fading in transitions, but it's also useful for things such as
side-chain effects, applying audio effects to entire scenes, applying
scene-specific audio filters on sub-sources, and other such
possibilities.
- The secondary issue that needed fixing was audio buffering.
Previously, audio buffering was always a fixed buffer size, so it
would always have exactly a certain number of milliseconds of audio
buffering (and thus output delay). Instead, it now dynamically
increases audio buffering only as necessary, minimizing output delay,
and removing the need for users to have to worry about an audio
buffering setting.
The new design makes it so that audio from the leaves of the scene graph
flow to the root nodes, and can be intercepted by parent sources. Each
audio source handles its own buffering, and each audio tick a specific
number of audio frames are popped from the front of the circular buffer
on each audio source. Composite sources (such as scenes) can access the
audio for child sources and do custom processing or mixing on that
audio. Composite sources use the audio_render callback of sources to do
synchronous or deferred audio processing per audio tick. Things like
scenes now mix audio from their sub-sources.
pkv