Samba 4 + Windows 10 time synchronization issues

Leave a reply

Where does ListeningThread -- Recvd 52 of 48/68 bytes come from?

If you follow the instructions for setting up Samba 4 AD DC for time synchronization, ntpd (coming out of Debian’s ntp package at some version 4.2.8) should just work.1

I came to this discovery after giving up and discarding my /etc/ntp.conf. Suddenly, after restarting ntpd and running w32tm /resync, things just worked. It’s not the software that’s broken — it’s me that was crazy.

The packet was now 110 bytes in Wireshark (68 of which was data). This was a stark improvement over seeing a 94 byte packet (52 of which was data). C:\temp\ntpDebug.log 2 no longer contained this:

ListeningThread -- Recvd 52 of 48/68 bytes

Hoozah! Now I wanted to figure out what was causing ntpd to send 52b packets, and not either 48b or 68b packets.

Turns out that my restrict statements had unexpected side effects. For instance, Samba wiki-recommended config tries to unrestrict localhost using restrict 127.0.0.1. 3

But I wanted to do the same for IPv6 localhost, so I did restrict ::1. This seems to have greatly confused ntpd.

The way out?

restrict -4 127.0.0.1
restrict -6 ::1

Second mistake was restrict 10.10.10.0 mask 255.255.255.0. It didn’t specify that mssntp should be enabled. For good measure I threw in -4:

restrict -4 10.10.10.0 mask 255.255.255.0 mssntp

Given that Samba config doesn’t recommend any special allowlisting for my internal IP range, I’ll just remove this line completely; the default restriction from the wiki should cover everything clients need to do anyway:

# Access control
# Default restriction: Allow clients only to query the time
restrict default kod nomodify notrap nopeer mssntp

Moral of the story? ntpd seems to be awfully sensitive to restrict statements. If w32time service complains or breaks in some way, be sure to remove the statements bit by bit, or make sure IPv4 and IPv6 statements don’t stomp over each other.

  1. Granted, I needed to modify the path to the socket to say /var/lib/samba/ntp_signd/ instead of /usr/local/samba/var/lib/ntp_signd/, but otherwise it just worked. ↩
  2. That file was created using w32tm /debug /enable /file:C:\temp\ntpDebug.log /size:102400 /entries:0-300 which I found somewhere online. ↩
  3. Apparently, passing no restrictions at all after the address simply means “unrestrict these peers”. ↩

Fixing "Use of undefined constant jq_syntax_htmlentities"

Leave a reply

I upgraded the PHP version backing my WordPress setup to PHP7.3. Of course, something had to go wrong:

Warning: Use of undefined constant jq_syntax_htmlentities - assumed 'jq_syntax_htmlentities' (this will throw an Error in a future version of PHP) in ...... wp-content/plugins/jquery-syntax/jquery-syntax.php on line 37

There’s no updates for jquery-syntax plugin that I’m using. Let’s assume that the author will fix this if they ever release an update.

For the time being, opening jquery-syntax.php manually and editing jq_syntax_quote to replace passing what looks like a function object (is it in PHP? I can’t be bothered to check) with a string when invoking preg_replace_callback() does the trick:

function jq_syntax_quote($content) {
        $content = preg_replace_callback('/<(pre)(.*?)>(.*?)<\/pre>/imsu','jq_syntax_htmlentities', $content);
        $content = preg_replace_callback('/<(code)(.*?)>(.*?)<\/code>/imsu','jq_syntax_htmlentities', $content);

        return $content;
}

Note how jq_syntax_htmlentities received single-quotes around it.

Detecting location of Visual Studio 2017 and 2019

Leave a reply

A few years ago, I wanted to use Bazel to build an old project of mine using Visual Studio 2017 Community.

Thing is, I don’t install most of MSVS on my system partition, as it’s on a relatively tiny SSD. So, where do I (portably) find it? And how do I set BAZEL_VS (per their documentation) to the correct path?

As I’m much more comfortable writing Bash scripts than writing old DOS-style Batch scripts, and as I’m going to have msys64 (or at least msys32) on my machines, clearly the answer is to write the resulting launcher in Bash. There are downsides to this approach, but it’s doable.

Do you even need this for Bazel?

No. I just tried running bazel-2.0.0-windows-x86_64.exe with my project. It found MSVS2019 out of the box both in cmd.exe and in the MSYS64 shell.

Unfortunately, I have a genrule that depends on bash, which didn’t work in cmd.exe even if I specified BAZEL_SH (missing binary for basename). And when I didn’t pass BAZEL_SH while running under MSYS64, it had trouble finding SDL/SDL.h presumably because the aforementioned genrule didn’t work.

Autodetection of MSVS2019 works, just set PATH Windows envvar to include msys binaries, and set BAZEL_SH Windows envvar to the output of cygpath -w /usr/bin/bash or cygpath -w /bin/bash.

You may want this for non-Bazel uses.

Finding Microsoft Visual Studio 2017

Assuming Bazel 2.0.0 is installed in C:\bazel under the default filename from the release on Github, this does the trick:

#!/bin/bash
# Based on https://superuser.com/a/539680 + searching around registry.
# Finds MSVS2017.
# Slashes when invoking 'reg query' are replaced with dashes because MSYS2.
VSPATH="$(cygpath -u "$(reg query 'HKLM\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\SxS\VS7' -v 15.0 | tail -n+3 | head -n1  | awk '{for (i=3;i<=NF;i++) {printf "%s ",$i;};}' | sed 's/ *$//')")"

export BAZEL_SH="$(cygpath -w /usr/bin/bash)"
export BAZEL_VS="$VSPATH"

/c/bazel/bazel-2.0.0-windows-x86_64.exe $@

Finding Microsoft Visual Studio 2019

Unfortunately this is broken as of MSVS2019. See the explanation why that registry key is not used anymore.

The new situation is better: there’s a tool that does get installed in a well-known path: %ProgramFiles(x86)%/Microsoft Visual Studio/Installer/vswhere.exe.

The problem is merely that Bash doesn’t like parentheses in environment variable names, so ${ProgramFiles(x86)} won’t be acceptable. However, it was visible in env output! Other people have spotted the same thing, but as their writings talk about PROGRAMFILES(X86) in all caps, I’ve concluded that maybe it’s better to add the I flag to their sed-based solution.

#!/bin/bash
# Let's use VSWhere:
PROGRAMFILES_X86=$(env | sed -n s,'^PROGRAMFILES(X86)=',,pI) # because parens are not valid envvar names: http://mingw-users.1079350.n2.nabble.com/msysGit-bash-and-PROGRA$
VSWHERE=${PROGRAMFILES_X86}/Microsoft\ Visual\ Studio/Installer/vswhere.exe
VSWHERE="$(cygpath -u "${VSWHERE}")"

# Finding VC: https://github.com/microsoft/vswhere/wiki/Find-VC
VSPATH=$("${VSWHERE}" -latest -products '*' -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath)

export BAZEL_SH="$(cygpath -w /usr/bin/bash)"
export BAZEL_VS="$VSPATH"

/c/bazel/bazel-2.0.0-windows-x86_64.exe $@

VSCode

The problem with using Bazel integration into VSCode is that it wants a path to Bazel. But Bazel wants to have BAZEL_VS set.

Maybe I could work around this by writing a tool in a compilable language, then have it pass on all CLI arguments to actual bazel.exe, while also setting BAZEL_VS. Or maybe Bazel supports some settings in .bazelrc. I didn’t care to check. Instead, I’ve used msys2_shell.cmd as the ‘shell’ for the integrated VSCode terminal. I’m used to manually running Bazel and I don’t care enough about interactive debugging. For now, I just don’t care enough to properly integrate Bazel into VSCode on Windows (though it would be neat).

Why even bother with Windows?

I need to trivially compare networking traffic between my project and a third-party program. The third-party program is written for Windows; it would likely work on other platforms under Wine, but I don’t care enough to explore whether I can make a third-party binary patcher for that program work under Wine.

My program still works on Linux, built whether with Bazel or using Makefiles; but the deprecated SDL 1.2 was recently broken on macOS, my program won’t work on macOS.

Getting this to work is also a nice exercise.

Anno 1404 crashing when clicking on Quickstart

Leave a reply
Regional Settings on Windows 10

Regional Settings on Windows 10. Making the countries match fixes Anno 1404.

I finally found the solution to the issue on the GOG.com forums. Nothing obvious like setting admin privileges helped.

No, it’s more insidious.

Apparently when you go to regional settings, both the “Country or Region” and the “Regional Format” values must match the country.

I’ve had these set to “Ireland” + “English (Europe)”. Before this, I likely had it set to “Ireland” + “Croatian (Croatia)”.

Changing the values to “Ireland” + “English (Ireland)” resolved the issue.

I can finally play the game years after buying it. This was done on Windows 10 1903 (build 18362.10012).

Time for FOSDEM 2019

Leave a reply

Previously I thought I would not head to Brussels this year, but it turned out I will. Let me know if you’d like to meet. I’m not sure if I’ll go to the beer event.

This year I hope to meet up with people interested in fediverse-related interop. Here’s a meetup thread.

I’ve also created an XMPP chatroom for meeting fediverse folks, in case realtime chat is needed; see xmpp:fediverse@muc.badc0de.net?join1. There’s a small web UI at https://badc0de.net/fosdem/2019/fediverse/2 and an IRC frontend3 at irc://irc.badc0de.net:6667/#fediverse; please join #fediverse manually. No guarantees about reliability of the XMPP server, of the HTTP proxy4, of the web UI, or of the IRC frontend. 🙂

On a related note, it’s disappointing that there’s a social fragmentation between Matrix, fediverse and XMPP, but eh. It would have been nice if ActivityPub was (like OStatus protocol suite before it) based around XML and Atom; it would have made pushing for supplanting the under-utilized XEP-0277 and its urn:xmpp:microblogging:0 with something that had a model of favoriting and commenting similar to fediverse.

Alas, to be useful now would probably require imagining ActivityPub 2.0’s model and vocabulary over XML, and use Atom for posts and activities. shrug

  1. This is running Prosody. ↩
  2. This is using Converse.js. ↩
  3. This is running the ultra-tiny MUC2IRC, written in Objective-C using ObjFW. ↩
  4. This is the Punjab BOSH connection manager running behind Haproxy running behind nginx running in a Mesos cluster managed by Marathon. 🙂 ↩

Using PostgreSQL row commit IDs and live stats

Leave a reply

Once again, it’s time to write down some notes for personal use. Let me know if what I’m jotting down here is particularly wrong.

PostgreSQL can tell you what are the commit IDs that touched a particular table. You query the system column named xmin.

prosody=> select xmin from prosody limit 5;
  xmin
--------
 212236
 770460
 770460
 967052
   1493
(5 rows)

You can also ask for the timestamp when this commit was created; however, that requires you to start tracking timestamps for each commit ID.

prosody=> select pg_xact_commit_timestamp(xmin), * from prosody;

ERROR:  could not get commit timestamp data
HINT:  Make sure the configuration parameter "track_commit_timestamp" is set.

This seems to be rather useful for built-in tracking of the modification timestamp and for etags. If the database backend is well structured, it may be possible to structure queries in such a way to quickly check when the results were last modified, and help the web frontend avoid serving and requesting the results. I don’t have a clear way to do it yet, but while I don’t think impact on the database will be significant, it may help shave off some serving bytes or requests to other backends.

Commit IDs also roll over after 32bit, so their use on a high traffic site needs to be closely considered. Then again, by the time you have over four billion writes, your caches will probably otherwise expire anyway.

Table pg_stat_activity is interesting and lets you see the transactions and even queries in flight. This’ll be slightly messy, but click ‘view raw code’ to see the original formatting, and scroll around a bit.

$ sudo -u postgres psql
psql (10.1, server 9.6.10)
Type "help" for help.

postgres=# SELECT pid, query FROM pg_stat_activity;
 pid  |                                                                                                                                                                                                              query                                                                                                                                                            
------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 8968 | SELECT o0."id", o0."token", o0."refresh_token", o0."valid_until", o0."user_id", o0."app_id", o0."inserted_at", o0."updated_at" FROM "oauth_tokens" AS o0 WHERE (o0."token" = $1)
...
...
 9869 | SELECT pid, query FROM pg_stat_activity;
(12 rows)
postgres=# SELECT * FROM pg_stat_activity;
 datid  |   datname   | pid  | usesysid | usename  | application_name | client_addr | client_hostname | client_port |         backend_start         |          xact_start           |          query_start          |         state_change          | wait_event_type | wait_event |        state        | backend_xid | backend_xmin |                                                                                                                                                                                                              query                       
--------+-------------+------+----------+----------+------------------+-------------+-----------------+-------------+-------------------------------+-------------------------------+-------------------------------+-------------------------------+-----------------+------------+---------------------+-------------+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 146762 | yyyyyyy_dev | 8968 |   146761 | yyyyyyy |                  | 127.0.0.1   |                 |       47545 | 2018-11-08 23:44:52.761448+00 |                               | 2018-11-08 23:50:33.742608+00 | 2018-11-08 23:50:40.75014+00  |                 |            | idle                |             |              | SELECT u0."id", u0."bio", u0."email", u0."name", u0."nickname", u0."password_hash", u0."following", u0."ap_id", u0."avatar", u0."local", u0."info", u0."follower_address", u0."last_refreshed_at", u0."inserted_at", u0."updated_at" FROM "users" AS u0 WHERE (u0."id" = $1)
....
  16391 | prosody     | 8833 |    16386 | prosody  |                  | 10.0.AA.AAA |                 |       51490 | 2018-11-08 23:44:30.819644+00 | 2018-11-08 23:50:28.826344+00 | 2018-11-08 23:50:28.826344+00 | 2018-11-08 23:50:28.826362+00 |                 |            | idle in transaction |             |              | BEGIN
  12409 | postgres    | 9869 |       10 | postgres | psql             |             |                 |          -1 | 2018-11-08 23:48:29.262371+00 | 2018-11-08 23:50:41.487958+00 | 2018-11-08 23:50:41.487958+00 | 2018-11-08 23:50:41.487961+00 |                 |            | active              |             |      1606549 | SELECT * FROM pg_stat_activity;
(12 rows)