shards mvr

  • Q&D cmd-line to show dependency graphs for `brew outdated`

    ```
brew outdated -q | while read searchRoot; do
    echo 'checking for '$searchRoot'...'
    searchIterations=10
    outFile=/tmp/$(date +%Y%m%d-%H%M%S)_brew_${searchRoot}.gv
    fullGraph=$(brew deps --installed --dot --graph --include-requirements --for-each --annotate)
    nodeList="$searchRoot"
    for i in $(seq 1 $searchIterations); do
        nodeListNew=''
        newNodes=$(echo "$fullGraph" | egrep " -> \"($nodeList)" | cut -d\" -f2,4 -s | tr '"' '\n' | sort | uniq | tr '\n' "|" | sed 's/|$//')
        if [ -n "$newNodes" ] 
        then
            nodeListNew=$(echo "$nodeList"$'\n'"$newNodes" | tr '|' '\n' | egrep -v '^$' | sort | uniq | tr '\n' "|" | sed 's/|$//')
        else
            break
        fi
        if [ "$nodeListNew" = "$nodeList" ] 
        then
            break
        fi
        nodeList=$nodeListNew
    done
    potOut=$(echo "$fullGraph" | egrep "$nodeList" | sort | uniq)
    echo 'digraph {'"$potOut"'}' > "$outFile"
    dot -O -Tpdf "$outFile"
    open "$outFile".pdf
done
```

    Code tested on macOS 14.4, with the following extras: brew, graphviz (containing dot)

  • safety tip for using `tar` and `--option`s

    this will fail (it ignores the excludes - frustrating!):
    tar -cf /dev/null ./ --exclude-from=file-of-excludes

    this will work:
    tar -cf /dev/null --exclude-from=file-of-excludes ./

    (hint: put the options before the path to archive)

  • csvkit to import CSVs in a uniform format

    I’ve really been enjoying the power of csvkit to reliably munge CSVs into a uniform format - regardless the format from the provider (not only the order of columns, but also more complex transformations like whether debits are positive or negative):

    SQLALCHEMY_SILENCE_UBER_WARNING=1 csvsql --no-inference \
--query 'SELECT COALESCE("Transaction Date",""),"=-"||COALESCE("Amount (USD)",""),COALESCE("Category",""),COALESCE("Description","")||"||"||COALESCE("Merchant","")||"||"||COALESCE("Type",""),"Apple Card" \
FROM stdin' \
| csvformat -T | tail -n+2 | sort

    ref: https://csvkit.readthedocs.io/

  • open iftop for each (cfg’d) network interface:

    for i in $(ifconfig -lu); do if ifconfig $i | fgrep -v 'inet 127.' | grep -q "inet [0-9]"; then echo $i; fi; done | while read intfc; do osascript -e "tell application \"terminal\" to do script \"sudo iftop -i $intfc\""; done

    (Mac; could be adapted)

  • Partially-formed process, to convert RTF formatting (ex: lists) from clipboard / pasteboard, to paste into Markdown format (ex: git):

    osascript -e 'the clipboard as «class RTF »' | tr -d '«»' | sed 's/^data RTF //' | xxd -r -p | /usr/local/bin/pandoc -f rtf -t markdown

    (for Mac; could be adapted)

  • A sudo softwareupdate --install --all never got past Downloaded. 😕 No prompt (cmd-line or GUI) to restart & nothing in-process (ex: verification). I was watching logs, and saw nothing amiss. Solution was to ^C & run it again - with --restart too.

  • ugh: in a dir within iCloud Drive, replacing a regular file, with a symlink of the same name, apparently breaks sync’ing 😟 - and good luck trying to find that via brctl log -w. (only reason I knew it broke: a periodic check of sync status)

  • oh; weird: if you happen to have /tmp open in a Finder window, brew install wget fails with dir_s_rmdir - apparently due to a “.DS_Store” file, a few levels below /tmp

  • macOS Login Items, and other "Allow in Background"

    There are lots of Login Items and other “Allow in Background” items in macOS these days (especially if the Mac is managed…) so it can be useful to get a list of them all - especially when names shown in the GUI, are not clear. The dumpbtm tool is verbose - though generally all you need is this abbreviated list:

    sfltool dumpbtm | grep -E '^([[:blank:]]{0,1}[^[:blank:]]|[[:blank:]]*(Name|URL):)'

  • how to convert macOS clipboard RTF (ex: from Notes.app), to Markdown? this’ll get you 95% of the way there: osascript -e 'the clipboard as «class RTF »' | tr -d '«»' | sed 's/^data RTF //' | xxd -r -p | /usr/local/bin/pandoc -f rtf -t markdown

  • Apparently it's time for M1 users, to migrate their Homebrew config. Like right now.

    Drat; that was an unpleasant surprise:

    For some reason, when I tried brew upgrade today, the response was: Cannot install in Homebrew on ARM processor in Intel default prefix.

    Weird; it’s been working fine, via Terminal in Rosetta, on my M1, for quite some time. Last I saw, that was required, on Apple Silicon.

    I don’t see any warning logged from past “brew update” runs. And I couldn’t find any announcement or release note, that Homebrew is now ready for prime-time on M1 - but apparently someone felt it necessary to force the issue. :/

    So, since I depend on brew and several tools installed that way - including some automation - I dropped what I was supposed to be doing, to sort this out. Here’s what it took for me:

    1. Run brew bundle dump - this will create a Brewfile in the current directory.
    2. May or may not be necessary: Quit & relaunch Terminal, without Rosetta.
    3. Install Homebrew - let it go to the new dir, /opt/homebrew/ - AKA ${HOMEBREW_PREFIX}.
    4. Run brew dump - which uses the previously-created Brewfile, to (hopefully) install what you were using (as much as it can) in it’s new structure, in ${HOMEBREW_PREFIX}.
    5. As long as this succeeded, delete / archive the Brewfile. If not, debug…
    6. If you haven’t personalized $PATH etc.: Follow the directions (in the output above) for updating that.
    7. If you have (personalized $PATH etc.): Update that to also include ${HOMEBREW_PREFIX} (using the ouput as a guide) - note that there are some new environment vairables here, to consider adding to your shell’s config. You’ll likely want to make sure the new dir is before the old one (so any tools not yet updated are still accessible - and new tools take precedence over the now stranded ones).
    8. Update any needed configs that live in /usr/local (ex: for clamav), into the correct place in the new ${HOMEBREW_PREFIX}.
    9. Now find wherever you might have references to brew tools, which are hard-coded for /usr/local, and update instead, to ${HOMEBREW_PREFIX}. Suggestions of places to look: /etc/crontab, user-specific crontabs (ex: crontab -l), LaunchItems and such. And scripts that might be hard-coded for /usr/local. (Yes; hard-coding is “bad” - and sometimes useful to, for example, be sure to use the tool installed via brew, vs. a built-in tool by the same name.)
    10. Repeat on any other systems you need to.
    11. Set a reminder to clean out the stuff in /usr/local, once it’s no longer needed. (It might be sufficient to which each tool, and confirm when they’re all found in ${HOMEBREW_PREFIX} vs. the old /usr/local.)

    Whew! That seemed a bit more difficult than necessary - especially since it was (as far as I could tell) forced without warning.

  • quick way to get notified that your other Mac (maybe a server?) has some software updates available: swupd_remote_check.sh

  • A Hack to Index Notes, for Quicksilver

    First: A few presumptions; this is a hack:

    • You have at least some grasp of operating at the (Mac) command-line; ex: in Terminal.
      • For instance: ~ (tilde) is an abbreviation is for your homedir.
    • You know what Quicksilver is.
      • (Yes; Far as I know, it’s Mac only.)
    • Debugging your setup is up to you; at the moment, I have extremely limited availability to help.
      • Feel free to drop a note; on the off chance it’s possible, I will try to help.

    How to set it up

    1. A (bash) script to list the Titles of your Notes:
      • The code is in separate section below.
      • Save the script to ~/dump-note-titles.sh.
      • chmod it, to be executable: 
        chmod u+x ~/dump-note-titles.sh
      • Run it once, to seed the output file: 
        ~/dump-note-titles.sh
      • Open the ~/titles-from-notes-app.txt file which it generated.
        • It should have a list of the Titles of your Notes.
    2. Keep this up-to-date, via cron:
    3. Config QuickSilver to use this new “index”:
      • Add a Catalog entry, using File & Folder Scanner.
      • For the Path, select the ~/titles-from-notes-app.txt file generated by the script above.
      • For the Include Contents setting, select Text Lines.
    4. Config a new Quicksilver Action, to open Notes by title:
      • Open Script Editor.app.
      • Paste the code in - see the separate section below.
      • Save it, as open note by title.scpt (File format: Script), to your Quicksilver Actions directory.
        • Which is usually here: ~/Library/Application Support/Quicksilver/Actions/.

    How to use

    • Fire up Quicksilver.
    • Start typing some substring of a Note Title.
    • Select the Title you want from the results.
    • Select open note by title as the action.

    bash Script ~/dump-note-titles.sh

    #!/bin/bash
osascript \
	-e 'set outFilePath to (((path to home folder) as text) & "titles-from-notes-app.txt")' \
	-e 'tell application "Notes"' \
	-e     'set nameList to name of every note' \
	-e 'end tell' \
	-e 'set outFile to open for access file outFilePath with write permission' \
	-e 'repeat with theName in nameList' \
	-e     'write theName & return to outFile as «class utf8»' \
	-e 'end repeat' \
	-e 'close access outFile' \

    Applescript open note by title.scpt

    using terms from application "Quicksilver"
	on process text ThisClipping
		tell application "Notes"
			show note ThisClipping
	end process text
end using terms from

    (Yes; I do intend to put this in a repo, where it belongs.)

  • clean up sensitive (Mac) Preview files

    It’s handy to print things to Preview - maybe you want to save the “security” questions and (random) answers from a new site registration, to a secure place (like 1Password) and now you’re wondering where that temporary file is - with that sensitive info.

    So; fire up a terminal:

    find $TMPDIR -type f -mmin -1440 -iname \*.pdf\* -print0 | xargs -0 -L1 -t -I% mv -i % ~/.Trash/; open ~/.Trash/

    Notes:

    • start find with these options / arguments:
      • in $TMPDIR - your own user’s dir for temporary files
        • (which will automatically get cleaned up - eventually)
      • -type f - we want to find files only
      • -mmin -1440 - files modified in the last 1440 minutes (1 day); tweak as you like
      • -iname \*.pdf\* - files of with an extension of pdf*
      • -print0 - output results null-terminated (to handle “special” chars)
    • | - pipe results to xargs, to process them using these options:
      • -0 - handle null-terminated input
      • -L1 - process one item at a time (if any)
      • -t - show commands as they’re executed
      • -I% - in the ensuing command, replace % there, with the resulting filename/s
    • mv - move these files, with the following options:
      • -i - interactive prompt (instead of overwrite)
      • % - the source filename, substituted by xargs
      • ~/.Trash/ - destination dir; put them here
    • open ~/.Trash/ - open (in Finder), for your review

  • log4j is not the problem

    The central problem, is the enormous jenga tower that we’ve built:

    • Full of dependencies that virtually no one understands.
    • Dependencies generally chosen, for expedience.
    • Chosen by coders at all levels of in/experience.
    • Driven by “ship it now”, and “move fast and break things”.
    • With management rarely caring about risks - until one becomes a public crisis, which can no longer be ignored.

    Predictions:

    1. These crises will get both worse, and more frequent - bad actors (*), have noticed how vulnerable everything is.
    • (* Bad actors of all stripes, not “just” in tech - and some have state-level resources.)
    1. Even still, there will be little fundamental change in how we write & deploy code - because most of the people making these decisions haven’t felt it. Yet.
    2. The “powers that be” think they’re insulated from the pain. It may not be long til they find out how wrong they are.

subscribe via RSS