163 Commits
Author SHA1 Message Date
jpark37 f55021be90 graphics-hook: Bump Vulkan version
Vulkan loader blocks implicit layers older than application apiVersion.
2022-07-29 23:22:39 -07:00
jpark37 8a4352c1e5 graphics-hook: Relax Vulkan allocation strategy
Stop requiring DEVICE_LOCAL. May be needed for integrated GPUs.
2022-07-29 23:22:39 -07:00
jpark37 b7d630af6b graphics-hook: Update VkResult strings 2022-07-29 23:22:39 -07:00
jpark37 4c7c1854bd graphics-hook: Track DXGI status with counter
Previous approach could mark Present exit too early.
2022-07-29 21:54:28 -07:00
Eric Laberge dfb9bb74f7 graphics-hook: Avoid conflict between Vulkan and DXGI Present 2022-07-29 21:54:07 -07:00
jpark37 69ff026647 libobs, win-capture: Share window helper code
Add "ms_" prefix as makeshift namespace.
2022-07-23 17:41:34 -07:00
PatTheMav 9145e3063d win-capture: Fix subprojects not installing on incremental builds
When changing a subproject (e.g. graphics-hook), the generated binaries
are not installed without also building win-capture. This commit enables
the subprojects to be treated as "plugins" which are installed to rundir
and also the installation directory without any additional workarounds.
2022-07-13 15:52:13 -04:00
jpark37 df96e68ca6 cmake: Disable LNK4099 warning
We are very unlikely to ship PDBs for deps.
2022-07-11 22:07:10 -07:00
PatTheMav d42c98fa22 win-capture: Fix added resources not properly copied to rundir
The `add_target_resource` function uses unnecessary path components
when copying files to the rundir, resulting in files added that way
not to end up where OBS expects them to.

The generated binaries created by sub-targets also need to be copied
as part of `win-capture`'s data files, which was easy to fix as the
target exists before CMake switches into the subdirectories.
2022-03-30 11:10:54 +02:00
PatTheMav 49e9d49943 plugins: Update CMakeLists.txt for included plugins 2022-03-16 23:11:08 +01:00
jpark37 40cea9ca50 win-capture: Remove dependency on psapi.lib 2021-09-12 14:10:36 -07:00
Jim 84bf08c171 Merge pull request #4720 from jpark37/detours-integration
Finish Detours integration for remaining game capture APIs
2021-08-23 23:18:10 -07:00
kokole dd6dd4e104 win-capture: Fix D3D12 third party overlay capture
Doesn't make sense to grab the oldest frame when not capturing overlays,
because the overlay was already rendered in it. This will grab the
latest frame (before the overlay is rendered).
2021-08-17 08:24:57 -07:00
jpark37 0f2c1447ab graphics-hook: Fix D3D11On12 usages
Use D3D12_RESOURCE_STATE_PRESENT for CreateWrappedResource arguments. We
should assume the application will set the backbuffer to PRESENT, and
expect that to be preserved across the Present() call.

Remove unnecessary calls to ReleaseWrappedResources.

Clean up (void**) casts with modern patterns.
2021-08-15 15:58:09 -07:00
jpark37 0ffcb66526 graphics-hook: Add Detours include dir 2021-05-19 19:52:49 -07:00
jpark37 57f9c61069 win-capture: Remove custom function hooking
Detours seems to be more stable.
2021-05-18 12:51:01 -07:00
jpark37 452ee646e1 graphics-hook: Use Detours for function hooking
Helped D3D12 path recover from window mode changes, so we assume it
should be more stable across the board for all API hooks.
2021-05-18 12:51:01 -07:00
jpark37 492f4c05e1 graphics-hook: Remove unused header 2021-05-18 12:51:01 -07:00
jpark37 924d9a7d9c win-capture: Remove D3D12 fix toggle
The new D3D12 path seems stable enough. Remove the old path.
2021-05-18 12:47:25 -07:00
jpark37 c6cee82c92 graphics-hook: Try multiple D3D12 queues
Keep trying queues until one of them passes D3D11On12CreateDevice.
2021-05-17 02:28:11 -07:00
jpark37 5290326b77 graphics-hook: Do not persist device unnecessarily 2021-05-17 02:28:11 -07:00
jpark37 7439e1c4b8 graphics-hook: More logging to help debugging 2021-05-17 02:28:11 -07:00
jpark37 d9d87e6502 graphics-hook: Kill early return 2021-05-17 02:28:11 -07:00
jpark37 c160a6ff3e graphics-hook: Give up on DXGI swap chain
If a swap chain hasn't been seen in a while, assume it no longer exists,
and reset the capture.
2021-05-17 02:28:11 -07:00
jpark37 3412f6fece graphics-hook: Reduce variable scopes 2021-05-17 02:28:11 -07:00
jpark37 32f9fc120a graphics-hook: Prevent recursive free
Make sure to set data.swap to null before calling data.free().
Otherwise, we may call data.free() again in the Release() hook.
2021-05-17 02:28:11 -07:00
jpark37 b899f06877 graphics-hook: Fix potential D3D12 device leak 2021-05-17 02:28:11 -07:00
jpark37 2e7b0e3340 graphics-hook: Use Detours for D3D12 hook
Will attempt to fully use Detours after 27 ships.
2021-05-17 02:28:11 -07:00
jpark37 4de5863d07 graphics-hook: Link Detours library 2021-05-17 02:28:11 -07:00
jpark37 2472529520 win-capture: Restore GL capture deduplication
Previous fix attempted to place swap_recurse count in gl_data struct,
where it could be unexpectedly memset to 0. Separate member from struct,
and make thread-local for additional safety.
2021-03-31 02:35:22 -07:00
jpark37 17cb0acdd7 graphics-hook: Fix build without COMPILE_D3D12_HOOK
Just need to forward declare D3D12 type.
2021-02-14 22:20:14 -08:00
jpark37 23e9d4153e win-capture: D3D12 swap chain queue usage
Attempt to schedule shared texture copies against the command queue that
the game's swap chain uses to try to reduce artifacts. The heuristics
for obtaining the queue are not perfect, so provide a toggle to use the
previous behavior.
2021-02-14 02:06:02 -08:00
jpark37 535f6b0adc win-capture: Typeless game capture textures
Modify game capture shared textures to be typeless if they could
potentially need SRGB and non-SRGB views in the future.

These capture APIs have been updated: D3D 10/11/12, Vulkan.

D3D8 capture does not use shared textures.

D3D9 and GL interop do not support typeless textures.

The new game capture DLL should be compatible with old versions of OBS.

Also removed a lot of dead code around pointless SRV/RTV support.
2021-01-11 14:18:12 -08:00
jpark37 013dd5a7a3 win-capture: Fix our own Vulkan spec violation
Pass valid instance when looking up VkCreateDevice.
2020-11-14 12:03:36 -08:00
jpark37 dad861b036 win-capture: Fix D3D leaks on swap chain release
For game capture, hook DXGI release function to release D3D objects if
the related swap chain is also being destroyed.

An added bonus is that the game capture hook will handle swap chain
recreation for applications that don't use ResizeBuffers.
2020-10-13 18:47:35 -07:00
jpark37 9ba768837b win-capture: Violate Vulkan spec for compatibility
Rivatuner Statistics Server calls vkGetInstanceProcAddr for
vkCreateDevice with a NULL instance. According to spec, this should
return NULL, but RTSS will dereference this and crash.

This change returns function pointers for every intercepted function
regardless of instance, which is enough to keep RTSS alive.
2020-10-03 22:41:11 -07:00
jpark37 7cf148e013 win-capture: Improve game capture messages
Tag debug output in the game process with "[OBS]" and mention that
graphics-hook.dll is ours.
2020-09-08 09:55:46 -07:00
jpark37 94475b8795 win-capture: Fix Vulkan crash on minimize restore
Set uninitialized data members to avoid crash coming back from minimize.
2020-09-01 20:20:59 -07:00
jpark37 7a66466efc win-capture: Add Vulkan instance creation fallback
Fall back to original arguments if our 1.1 API request doesn't work.
2020-09-01 18:59:13 -07:00
jpark37 e905b4cb6e win-capture: Vulkan variable naming consistency 2020-09-01 18:56:14 -07:00
Jim 6efd70fc01 Merge pull request #3260 from jpark37/vulkan-cts-hack-remove
win-capture: Remove Vulkan CTS workaround
2020-08-27 10:18:08 -07:00
jpark37 a713a5b433 win-capture: Robust Vulkan swap chain handling
Not all surfaces support VK_IMAGE_USAGE_TRANSFER_SRC_BIT. Only
VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT is required by spec. Add fall back.
2020-08-23 11:23:12 -07:00
jpark37 9d177d0969 win-capture: Remove Vulkan CTS workaround
CTS has been fixed, so the workaround is no longer necessary.
2020-08-07 22:58:49 -07:00
jpark37 04b9b9fea3 win-capture: Make Vulkan frame data local to queue
Prevents multiple queues in the same family from racing on frame data.

Removal of vk_family_data also fixes broken capacity calculation. Two
queue families can share the same index if one is "protected-capable"
and the other isn't.
2020-07-26 13:33:01 -07:00
jpark37 e6896d79e3 win-capture: Hide Vulkan linked list internals 2020-07-26 13:32:53 -07:00
jpark37 44fb61b1f2 win-capture: Improve Vulkan hook stability
Make fixes mostly to stabilize CTS. Tested on Windows with RTX 2080 Ti.

Use separate linked-list maps for devices and queues.

Process queues upfront in OBS_CreateDevice, and not lazily in
OBS_GetDeviceQueue.

Handle failed node allocation for OBS_CreateInstance, OBS_CreateDevice,
and OBS_CreateSwapchainKHR.

Handle VK_NULL_HANDLE for OBS_DestroySurfaceKHR and
OBS_DestroySwapchainKHR.

Add temporary code path CTS_WORKAROUND to deal with CTS sharing surfaces
between instances.

Fix return value for OBS_DestroyInstance.

Add missing _freea calls, required for _malloca.

Make sure the queue passed to OBS_QueuePresentKHR supports transfer to
safely call vkCmdCopyImage.

Update GetInstanceProcAddr/GetDeviceProcAddr to conform closer to spec.

Remove extra macros because I dislike indirection more than verbosity.
2020-07-24 09:30:05 -07:00
jpark37 afbad3b337 win-capture: Remove fixed-size Vulkan arrays
Use linked lists in place of fixed-size arrays in an attempt to fix
behavior when programs generate lots of non-resource objects.
2020-07-10 20:41:00 -07:00
Jim 5728df6cef Merge pull request #3149 from jpark37/vulkan-32bit
win-capture Fix 32-bit Vulkan capture
2020-07-10 20:39:13 -07:00
Jim 6c37f6faea Merge pull request #3113 from jpark37/vulkan-swap-decouple
win-capture: Decouple swap and frame indices
2020-07-10 20:15:38 -07:00
jpark37 7772a56a97 win-capture: Fix 32-bit Vulkan capture
Vulkan loader needs pure name, which needs manual fix-up on 32-bit.
2020-07-09 22:10:34 -07:00