hs.touchbar - Add Extension for Touch Bar Support #1078
Comments
|
This might also be of interest: |
|
@cmsj mentioned https://github.com/bikkelbroeders/TouchBarDemoApp which also has useful code to steal I bet |
|
I'm not exactly sure what we could bring to the table that those apps aren't already covering. Does anyone have something in mind? |
|
I took a look at TouchBarDemoApp last night... When used without an iPad/iPhone as the "display", the bar appears right above the mouse cursor... I guess that makes sense since you have to use the mouse pointer to select the button you want, but from a visual standpoint, I might like more control over where the bar appears and maybe a way to make it persistent (both of which could more easily be done with the existing app, if they're accepting feature or pull-requests). And while the PeerTalk framework looks interesting for communicating with physically attached iPad/iPhone devices, without having an iPad application to go with Hammerspoon, it's at best an interesting third-party module... with sideloading required to get even that, I'd much rather see a web/websocket interface to Hammerspoon explored that any web browser/web-enabled-app on any device could potentially use over the network and not require the physical connection. It think we have all of the pieces needed for this already. I tend to agree that I'm not sure what we can add to this. I'm much more interested in the idea of creating our own touch bars... it looks pretty straight forward to create a "Hammerspoon" touch bar, but I'm not sure yet if we can add anything to the touch bars of other applications. Just my 2 cents... |
|
@asmagill - I definitely think you should look into creating a proper Hammerspoon Touch Bar. I'd love to be able to press a button and instantly get up the Console. However, why I'm interested in you porting the TouchBarDemoApp to Hammerspoon, is that in Final Cut Pro for example, Apple's added some features to the Touch Bar that don't really exist in the native Desktop app. There's a few button's in the Final Cut Pro touch bar, that users would LOVE to have on their older Mac laptops and desktops, even if it's controlled via a mouse. Why I want this functionality in Hammerspoon rather than a 3rd party App, is that then I could control when the Touch Bar appears, it's position on the screen, and in a perfect world, it would be awesome if you could programatically manipulate the touch bar (i.e. in the case of Final Cut Pro, I'd only show the user Touch Bar options that don't already have a easy-to-click Desktop app equivalent. For me, bringing this functionality into Hammerspoon is all about control. I want to be able to control the Touch Bar simulator. One day I would LOVE to see an iOS Hammerspoon App (which you "control" via the Desktop scripts) - but that's a big engineering challenge, and a discussion for it's own thread! Any questions let me know! |
|
@asmagill for injecting HS into other apps' TouchBars, we'd have to use code injection. I have done some work on this (see cmsj@79f76bc ) but it's extremely ugly, requires building a 32bit version of everything, and for some weird reason I had to completely butcher parts of LuaSkin's objects, to make it work. I'm afraid I'm not convinced that we should implement a TouchBar feature. I can see how it might be fun/useful, but I think the amount of work involved would outweight the benefits. If someone else wants to work on it, please don't let me stop you :) As for a companion iOS app, it's something I've looked at before, there's an Issue open - #412 - and some code exists already, but I've not had much time to work on it. |
|
@cmsj, I suspect you're right about needing code injection... the little I have read so far talks about needing to put the NSTouchBarItems into the active NSTouchBar of the Application's responder chain, something I don't think we can access without code injection... I may toy with creating a Hammerspoon touchbar at some point (conceptually it seems similar to the toolbar code already in hs.webview.toolbar) since we can always programmatically make Hammerspoon the active application, then switch it back once "whatever" has been touched/clicked... but I tend to agree, short term I don't see it as a priority. @latenitefilms are you certain the actions aren't available any other way? Apple guidelines strongly suggest that developers should never put something into the touch bar that can't be accessed another way since right now very few actually have the touch bar. (Of course it wouldn't be the first time a developer has ignored Apple's guidelines, but... this soon after its introduction?) I haven't explored it with axuielement yet though, so that may offer possibilities. |
|
@asmagill - It's not so much that the actions aren't avalibile any other way, however for example, when you have a MacBook Pro with a Touch Bar and are using Final Cut Pro (which is made by Apple), it will display a mini-timeline on the Touch Bar, allowing you to very easily and quickly jump between different sections of the timeline with the tap of your finger. This feature doesn't really exist as part of the standard desktop app - you can't "jump" to a specific timeline section in one click, you just have to use the normal scrollbars. This "mini timeline" alone would be REALLY handy to desktop users WITHOUT a Touch Bar. I could then use Hammerspoon's amazing GUI Scripting abilities to keep track of Final Cut Pro's window positions, and "move" the Touch Bar Simulator so that it basically "feels" like it's a native part of the App. So although the 3rd Party Touch Bar tool is interesting - all it does it just display the Touch Bar on your screen. What I would like to do is CONTROL how that Touch Bar Simulation is positioned, and control WHEN it's visible. I'll download the TouchBarDemoApp and have a play - but I don't think I'm smart enough to "port" it across to Hammerspoon sadly.
|
|
@latenitefilms hope you don't mind... I needed a diversion and came up with this: https://github.com/asmagill/hammerspoon_asm/tree/master/touchbar It just shows and hides the touch bar (and a few additions I wanted like automatic dimming and more control over placement) -- you'll need to wrap it with a hotkey or eventtap watcher to toggle it as you prefer, but it seems to work well for me so far. I hope this doesn't dampen your interest in possibly contributing something in the future... it's really not that hard once you've done it a time or so... you can check out my walkthrough at https://github.com/asmagill/HS_ModuleWalkthrough/ if you like... it's a little dated, and I would trust the sample code a little more than I would the readme (as an example, we're importing the frameworks as modules now, so @import syntax needed to be added to the Makefile, and while I've checked that the code itself still compiles, I haven't had a chance to update that readme yet), but conceptually its still accurate. I'm hoping over the December holidays to do a set of Wiki documents going into detail with LuaSkin and our object helpers, and that should help as well. |
|
@asmagill - You're AMAZING! Thank you so much!! I need to update macOS before I can play, but will do this tonight, and give it a proper test run tomorrow! I'll also definitely look into the Module Walkthrough properly at some point too! Thanks again! I'm hoping to get back to doing some HS coding over the next few days, as I've been flat out the last couple of weeks. |
|
I should note that I'm not completely happy with the garbage collection during a reload of Hammerspoon... Essentially the code creates a window and then streams the video from the touchbar into it using undocumented/private functions to get at the stream... since the sample app does one thing and one thing only, it just relies on the normal application teardown to clean things up when you quit it... we have to be a little more careful (well we should be, since we support reloading without quitting). Usually any module which uses a window (hs.drawing, hs.webview, my canvas/enclosure modules, etc.) includes a I did add code to stop the stream (the sample app didn't), but until I can dig a little deeper, anything else I add results in a crash, so... if you reload Hammerspoon a lot (without completely quitting it and then re-opening the application from the Finder), then I strongly suspect that this will leak memory... slowly, but still... It's worked well for me for the last 24 hours (it's interesting to see what different applications put into the bar), but until I am more comfortable with the garbage collection portion, I wouldn't even consider adding this to core... and even then, since the undocumented calls are pretty new (we do already use some in Hammerspoon, but the ones I'm aware of have been around since at least 10.8 or longer and haven't changed in all that time)... well, I'll go with the consensus once I think it's ready. |
|
@asmagill - This is really helpful and awesome - thank you! I'm wondering - is there a way to "crop" the Touch Bar window? A lot of the Touch Bar icons on the far left of the "screen", are generic macOS functions (like volume, brightness, Siri, etc.) - which I don't really need. Thoughts? Is there also a way to "scale" the Touch Bar window? Currently the icons are fairly large - I wonder if there's a way to make everything just a little bit smaller? The ability to drag the Touch Bar around the screen with your mouse could also be a handy addition? I've had a quick look through your Objective-C code to see if I could understand it, and I have absolutely no idea, so sadly these are additions I could definitely not do myself. Thanks as always for your amazing code and support!
|
|
As to the system icons, that is determined by the setting in the Keyboard panel of the Keyboard System Preferences:
As to resizing, at present, no, I don't think we can resize it... it's basically a streamed view of the touchbar display into a window and the stream is coming from non-published methods... I just don't know enough about the video source to manipulate it at present. |
Perfect, thanks!
No worries! Thanks @asmagill! |
|
@asmagill - Because the Touch Bar is context sensitive, one of the issues I have is that if the Touch Bar is at the bottom of the screen for example, but the thing I want to control is at the top of the screen, when I move my mouse from the top of the screen to the bottom, obviously the Touch Bar changes. It would be great if there was somehow to hold down a modifier key, to prevent the Touch Bar from updating. |
|
We have no control over what is presented in the touch bar itself -- all we are doing is streaming what the Mac would be showing in the touch bar if your Mac had a physical one. You can move the touch bar window closer to your desired target if the problem is that mouse movements over the window causes the touch bar contents to change, but that's the best we can do -- we have no control over what is actually in it. |
|
Shall we re-open this? If the code can be hammered into shape, maybe it should go in? |
|
@cmsj - I've been using @asmagill awesome Hammerspoon code in FCPX Hacks shortly after he released it, and it's been working great for me! Haven't noticed any major bugs or issues yet. |
|
I've added some additional methods and provided an example of how it might be used with an option to drag it into a new position without requiring console use. I'd still like to clear up the __gc issue properly, and I'd like to find a way to detect when the user is customizing the toolbar so we can force it to alpha = 1... it's annoying to have it auto-dim if you've set it too low while you're trying to decide what to drag in or out of it. Those two reasons, and of course more use-testing, would be my main reasons to hold off for a bit before adding this to core. @cmsj, since we're hoping to be able to figure out how to maybe add an item like the example you give in #1096, and I'd like to be able to create a full Hammerspoon touch bar for use when the Hammerspoon application is focused, what do you think I should name this when it's added? "hs.touchbar"? "hs.touchbar.viewer"? Other ideas? |
|
@asmagill - Awesome! Thanks so much! I'll test out this latest version tonight! I also agree it would be great to add a proper Hammerspoon Touchbar. I think hs.touchbar makes sense to control Hammerspoon's native Touchbar. Maybe hs._asm.touchbar should be renamed to hs.onscreenTouchbar? |
|
@asmagill - FYI: I just downloaded touchbar-v0.5.tar.gz, however when I try touchbar:movable() it gives me a "attempt to call a nil value (method 'movable')" error. |
|
Have you fully quit and restarted Hammerspoon? Whenever the objective-c portion of a module changes, it requires an actual quit and re-launch before the new version is loaded. |
|
@asmagill - Opps! Sorry, rookie error, I didn't know that - restarting has fixed! Thank you! |
|
It's caught us all... and it's a bit of a pain, really. It's because of the way Objective-C loads dynamic code... Objective-C intentionally "opens" a library twice so the traditional method of calling |
|
@asmagill - No worries at all! I should have really tried restarting before posting anyway. Loving the moveable (or movable as you call it!) Touch Bar! Very handy! I'm currently using setCallback as a way to know when the Touch Bar has moved (so I can save it's last position in settings so it can be restored during a restart) - however, I don't really get what you mean by using acceptsMouseEvents as a way to prevent mouse clicks in the touch bar from triggering the touch bar buttons? Are you thinking that rather than making the Touch Bar ALWAYS draggable, it should be used with a shortcut key held down or something? |
|
Normally when you click on the bar, the click is passed through and interpreted as if you had "touched" the touch bar... while technically movable is toggling the ability for a window to be moved by clicking in it's background, in truth what is being displayed is a video stream, so... there is no "background"... the mouse click is still getting passed through... try clicking on a button and holding it while dragging -- you'll see that not only does the window move, but the "button" gets toggled as well. I also found that some of the buttons which appear after a long "touch" (in Terminal, for example, if you touch one of the buttons for an extended period, it will show up some additional buttons for Search Term Display) were unusable when I guess I could have combined them and made movable automatically make the touch bar "view only" but... I don't like doubling up on things when its not strictly necessary because the touch bar is usable in the movable mode, just not fully, and I prefer having options (as long as they don't cause a crash, of course!) If you look at my example code in the Examples folder, I pair I can look at the documentation again tomorrow (it's almost 2 AM here, so I'm off to la-la-land in a bit!) and see if it needs some clarification. As to the spelling... I guess it's an American thing... my dictionary says both spellings are correct, but dropping the |
|
@asmagill - Legend, thanks so much! Go to bed! |
|
@asmagill re your question about namespacing, what about something like this:
There is quite a lot of API to cover with NSTouchBarItems, but I think we could cover a bunch of common ground pretty quickly - which is to say NSButtons that are either text or NSImage based. From there we can extend over time to cover things like the scrubbers, sliders, color pickers, etc. Apple has a pretty excellent example app which covers most of the different kinds of system controls. |
|
@asmagill - Also, is there a good way of programatically organising the order of new buttons? I've tried using |
|
Actually, ignore my last comment, sorting |
|
FYI: It seems the Touch Bar really only likes images that are 36px × 36px, so I can just "force" it to that using Related: https://developer.apple.com/macos/human-interface-guidelines/touch-bar/touch-bar-icons-and-images/ |
|
FYI: The Touch Bar REALLY doesn't like it when you feed it two identical |
|
If it wasn't made clear in the documentation (such as it is), duplicate identifiers are not allowed. Of course a better error should be returned, but this isn't a surprise. |
|
I just did it by accident - and it crashed Hammerspoon, so it would be good to just display a nice error message instead. It's awesome by the way - really LOVING this extension! |
|
It would also be good to be able to specify a width for a |
|
You're talking about it's horizontal width, correct? As I recall, I was having problems making that work -- it wants to expand to fit the available space based on the number of other elements... but I'll look into it again because it seems an obvious thing developers might want. The touchbar is the first visual component I've come across that requires you to use constraint based layout features... it's optional everywhere else and so far I've found it easier most of the time to avoid them. A future extension to the guitk manager may allow using constraint based layout for guitk, but my initial attempt made things much more confusing and didn't quite work as expected, so I backed off of it for now. For touchbar I don't think that will be an option. |
|
(ok, width is horizontal, I guess I meant physical width, as in display width) |
|
@asmagill - Ok, so this is a long shot... Is it possible to "steal" elements from an existing Touch Bar and "insert" them into Hammerspoon's modal touch bar? For example, QuickTime Player in Sierra has an awesome timeline/thumbnail navigator. Would it be possible to "steal" this Touch Bar element and insert it into a Hammerspoon modal Touch Bar when QuickTime Player is open, so that you can also add custom buttons next to it? Basically the reason being - I'd like to add custom buttons to applications - but I don't want to loose native functionality that's built into macOS's applications. Thoughts? |
|
@asmagill - I have "App Controls with Control Strip" selected in the Keyboard System Preferences, and have an icon in the control strip which enables my custom Touch Bar. It works great. However, sometimes when I press the X button on the far left of the Touch Bar, it will close my bar, but it won't restore the Touch Bar of whatever application I'm running. The only way I can get the applications "native" Touch Bar back is to restart Hammerspoon. Any ideas? |
|
@asmagill FYI: |
|
@asmagill The documentation says:
However, it looks like BetterTouchTool can now take over the entire Touch Bar, hiding the Control Strip for specific applications. Is this something that could be added to your Touch Bar module?
|
|
@asmagill - Another crash, maybe related to this extension? |
|
@asmagill - This may be of some interest: https://github.com/a2/touch-baer |
|
@asmagill - This might be of interest if you're using macOS 10.14 Mojave: |
|
@asmagill - FYI: |
|
@asmagill - FYI: Another one: |
|
Just saw this: 2019-04-01 15:36:30: ********
2019-04-01 15:36:30: 15:36:30 ERROR: prop: Error while notifying a watcher: NSInvalidArgumentException: +[NSTouchBar dismissSystemModalFunctionBar:]: unrecognized selector sent to class 0x7fff87bc78b0
(
0 CoreFoundation 0x00007fff2f7daded __exceptionPreprocess + 256
1 libobjc.A.dylib 0x00007fff5b8a6720 objc_exception_throw + 48
2 CoreFoundation 0x00007fff2f8580e5 __CFExceptionProem + 0
3 CoreFoundation 0x00007fff2f77ca60 ___forwarding___ + 1486
4 CoreFoundation 0x00007fff2f77c408 _CF_forwarding_prep_0 + 120
5 bar.so 0x0000000113843b64 touchbar_dismissSystemModalFunctionBar + 148
6 LuaSkin 0x000000010d467552 luaD_precall + 699
7 LuaSkin 0x000000010d463877 luaV_execute + 723
8 LuaSkin 0x000000010d4676d8 luaD_call + 64
9 LuaSkin 0x000000010d467719 luaD_callnoyield + 21
10 LuaSkin 0x000000010d452ced luai_objcttry + 28
11 LuaSkin 0x000000010d467ca1 luaD_pcall + 116
12 LuaSkin 0x000000010d46129c lua_pcallk + 236
13 LuaSkin 0x000000010d466aa3 luaB_xpcall + 115
14 LuaSkin 0x000000010d467552 luaD_precall + 699
15 LuaSkin 0x000000010d463877 luaV_execute + 723
16 LuaSkin 0x000000010d4676d8 luaD_call + 64
17 LuaSkin 0x000000010d467719 luaD_callnoyield + 21
18 LuaSkin 0x000000010d452ced luai_objcttry + 28
19 LuaSkin 0x000000010d467ca1 luaD_pcall + 116
20 LuaSkin 0x000000010d46129c lua_pcallk + 236
21 LuaSkin 0x000000010d44b551 -[LuaSkin protectedCallAndTraceback:nresults:] + 203
22 LuaSkin 0x000000010d44b606 -[LuaSkin protectedCallAndError:nargs:nresults:] + 55
23 usercontent.so 0x0000000113487af7 -[HSUserContentController userContentController:didReceiveScriptMessage:] + 331
24 WebKit 0x00007fff3e67628e _ZN28ScriptMessageHandlerDelegate14didPostMessageERN6WebKit12WebPageProxyERKNS0_13FrameInfoDataERN7WebCore21SerializedScriptValueE + 190
25 WebKit 0x00007fff3e62e8f0 _ZN6WebKit29WebUserContentControllerProxy14didPostMessageERN3IPC10ConnectionEyRKNS_13FrameInfoDataEyRKNS1_13DataReferenceE + 178
26 WebKit 0x00007fff3e631179 _ZN3IPC13handleMessageIN8Messages29WebUserContentControllerProxy14DidPostMessageEN6WebKit29WebUserContentControllerProxyEMS5_FvRNS_10ConnectionEyRKNS4_13FrameInfoDataEyRKNS_13DataReferenceEEEEvS7_RNS_7DecoderEPT0_T1_ + 277
27 WebKit 0x00007fff3e34f08f _ZN3IPC18MessageReceiverMap15dispatchMessageERNS_10ConnectionERNS_7DecoderE + 127
28 WebKit 0x00007fff3e5d6000 _ZN6WebKit15WebProcessProxy17didReceiveMessageERN3IPC10ConnectionERNS1_7DecoderE + 24
29 WebKit 0x00007fff3e31be48 _ZN3IPC10Connection15dispatchMessageENSt3__110unique_ptrINS_7DecoderENS1_14default_deleteIS3_EEEE + 130
30 WebKit 0x00007fff3e31e6e7 _ZN3IPC10Connection24dispatchIncomingMessagesEv + 731
31 JavaScriptCore 0x00007fff32d45047 _ZN3WTF7RunLoop11performWorkEv + 231
32 JavaScriptCore 0x00007fff32d452d2 _ZN3WTF7RunLoop11performWorkEPv + 34
33 CoreFoundation 0x00007fff2f740395 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
34 CoreFoundation 0x00007fff2f74033b __CFRunLoopDoSource0 + 108
35 CoreFoundation 0x00007fff2f723dd1 __CFRunLoopDoSources0 + 195
36 CoreFoundation 0x00007fff2f72337a __CFRunLoopRun + 1219
37 CoreFoundation 0x00007fff2f722c64 CFRunLoopRunSpecific + 463
38 HIToolbox 0x00007fff2e9b9ab5 RunCurrentEventLoopInMode + 293
39 HIToolbox 0x00007fff2e9b97eb ReceiveNextEventCommon + 618
40 HIToolbox 0x00007fff2e9b9568 _BlockUntilNextEventMatchingListInModeWithFilter + 64
41 AppKit 0x00007fff2cc74363 _DPSNextEvent + 997
42 AppKit 0x00007fff2cc73102 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1362
43 AppKit 0x00007fff2cc6d165 -[NSApplication run] + 699
44 AppKit 0x00007fff2cc5c8a3 NSApplicationMain + 780
45 libdyld.dylib 0x00007fff5c974ed9 start + 1
46 ??? 0x0000000000000001 0x0 + 1
)
2019-04-01 15:36:30: ******** |
|
hey guys, sorry if I'm asking in the wrong place, is It would be cool to shove my own buttons into the touchbar and customize everything with lua. I feel like the default apple touchbar options are pretty limited. I saw all kinds of apps putting animations and other useless stuff in the touchbar, but there are very limited choices that allow the kind of customization that we could do with the rich HS api. thanks |
|
Yes, Check out this We use it in CommandPost, and it works really well. FYI - CommandPost is a fork of Hammerspoon, with a bunch of custom Lua code to add additional features to the post production community. It's also open source, so you can check out how we use it. @asmagill's code currently hasn't been updated to work with Mojave and Catalina, but you can refer to this pull request for the fix. The only limitation we've run into is that it currently doesn't work with the 16-inch MacBook Pro. We currently don't have a solution for this yet. Any questions let me know. |
|
Oops, thanks, I missed the fact that there are multiple readme files in that repo, I just saw the main one that says that it's a virtual touchbar :) I saw a youtube video demonstrating how CommandPost manages the touchbar - it's really impressive. I saw comments about it not working in the 16-inch model, that's unfortunate. Thanks! |
|
I would still recommend trying it, because I don't actually personally own a 16-inch MacBook Pro, so I'm just going off other user reports - maybe you can give some better clues as to why it's not working? I'm hoping the genius that is @asmagill might also have a chance to another look at the code at some point to see if anything obvious stands out. The Touch Bar size on the 16-inch MacBook Pro is a different size, and uses a new T2 chip - so I'm guessing that's SOMETHING to do with it, but can't spot anything obvious in the code. |
|
Now that Apple has dropped the Touch Bar on the 14-inch and 16-inch MacBook Pro, I'm going to close this issue. We'll still keep using @asmagill's awesome extension in CommandPost, but this functionality probably doesn't need to make it's way to the Hammerspoon core anymore. |
Sorry - I've been out of action the last couple of weeks on an interstate film shoot.
Has anyone had a chance to look into the Touch Bar API as of yet?
It would be AWESOME if you could build something similar to this using Hammerspoon:
https://red-sweater.com/touche/
Thoughts?
The text was updated successfully, but these errors were encountered: