Samba AD issues are hard, hard, hard to fix

So it looks like I did something wrong at some point while setting up a domain and Samba4 is now broken for me.

Of course this had to happen after I spent time migrating my local account to the domain account. (No, it did not go as smoothly as the sources might lead you to believe.)

So I am understandably reluctant to reprovision the machine and go through that process again, breaking who-knows-what-else by breaking the NTFS ACLs formed since.

So yeah, I’ll use this post as an outlet for complaints about this breakage:

Mar  1 00:42:17 commander samba[12250]: [2015/03/01 00:42:17.752020,  0] ../lib/util/util_runcmd.c:317(samba_runcmd_io_handler)
Mar  1 00:42:17 commander samba[12250]:   /usr/sbin/samba_dnsupdate: Traceback (most recent call last):
Mar  1 00:42:17 commander samba[12250]: [2015/03/01 00:42:17.753033,  0] ../lib/util/util_runcmd.c:317(samba_runcmd_io_handler)
Mar  1 00:42:17 commander samba[12250]:   /usr/sbin/samba_dnsupdate:   File "/usr/sbin/samba_dnsupdate", line 612, in <module>
Mar  1 00:42:17 commander samba[12250]: [2015/03/01 00:42:17.753757,  0] ../lib/util/util_runcmd.c:317(samba_runcmd_io_handler)
Mar  1 00:42:17 commander samba[12250]:   /usr/sbin/samba_dnsupdate:     get_credentials(lp)
Mar  1 00:42:17 commander samba[12250]: [2015/03/01 00:42:17.754374,  0] ../lib/util/util_runcmd.c:317(samba_runcmd_io_handler)
Mar  1 00:42:17 commander samba[12250]:   /usr/sbin/samba_dnsupdate:   File "/usr/sbin/samba_dnsupdate", line 118, in get_credentials
Mar  1 00:42:17 commander samba[12250]: [2015/03/01 00:42:17.755084,  0] ../lib/util/util_runcmd.c:317(samba_runcmd_io_handler)
Mar  1 00:42:17 commander samba[12250]:   /usr/sbin/samba_dnsupdate:     creds.set_machine_account(lp)
Mar  1 00:42:17 commander samba[12250]: [2015/03/01 00:42:17.755797,  0] ../lib/util/util_runcmd.c:317(samba_runcmd_io_handler)
Mar  1 00:42:17 commander samba[12250]:   /usr/sbin/samba_dnsupdate: RuntimeError: (-1073741275, 'NT_STATUS_NOT_FOUND')

…uh, so the machine account is missing? What? How did that happen? Is it really missing?

# samba-tool user create COMMANDER$
New Password: #DUMMYPASSWORDHERE#
Retype Password: #DUMMYPASSWORDHERE#
ERROR(ldb): Failed to add user 'COMMANDER$':  - samldb: Account name (sAMAccountName) 'COMMANDER$' already in use!

Let’s try this, found on Samba’s wiki:

samba-tool dbcheck --fix --reset-well-known-acls

Hurray, an error has been fixed! But everything is still horribly broken.

Oh look! There’s a DC diagnostics tool shipping in Windows:

C:\Users\ivucica>dcdiag /s:ds.badc0de.net /v

Directory Server Diagnosis

Performing initial setup:
   * Connecting to directory service on server ds.badc0de.net.
   Ldap search capability attribute search failed on server ds.badc0de.net,
   return value = 52

Thanks, Microsoft, that’s helpful.

So I fiddled a bit and ended up with this:

C:\Users\ivucica>dcdiag /s:commander

Directory Server Diagnosis

Performing initial setup:
   Ldap search capability attribute search failed on server commander, return
   value = 81

C:\Users\ivucica>dcdiag /s:commander.ds.MYDOMAIN

Directory Server Diagnosis

Performing initial setup:
   Ldap search capability attribute search failed on server
   commander.ds.MYDOMAIN, return value = 81

No, passing /v did not help identifying either error 52 nor 81. But that 81 is mildly googlable. Wait, it’s mentioning LDAP… Is it even running?

Oh wait, Microsoft has another diagnostics tool (of course it does)

C:\Users\ivucica>nltest /dsgetdc:ds.MYDOMAIN force /gc
Getting DC name failed: Status = 1355 0x54b ERROR_NO_SUCH_DOMAIN

Of course there is no such domain, why would there be, am I right? tcpdump revealed that UDP packets on 389 were being rejected (and nothing is listening there). And connections to localhost were failing. So let’s look at help for samba-tool dbcheck. Huh. Let’s try this:

samba-tool dbcheck --fix --reindex --scope=base

And breakage begone!

C:\Users\ivucica>nltest /dsgetdc:ds.MYDOMAIN /force /gc
           DC: \\commander.ds.MYDOMAIN
      Address: \\10.0.99.150
     Dom Guid: b066b58f-6fa9-42d6-a45a-ABCDEFABCDEF
     Dom Name: ds.MYDOMAIN
  Forest Name: ds.MYDOMAIN
 Dc Site Name: DO-AMS1
        Flags: PDC GC DS LDAP KDC TIMESERV GTIMESERV WRITABLE DNS_DC DNS_DOMAIN
DNS_FOREST
The command completed successfully

Or not?

C:\Users\ivucica>dcdiag /s:ds.badc0de.net /v

Directory Server Diagnosis

Performing initial setup:
   * Connecting to directory service on server ds.badc0de.net.
   Ldap search capability attribute search failed on server ds.badc0de.net,
   return value = 52

Back to 52. And samba_dnsupdate is still broken, and the workstation cannot administrate the DC. Because, “The server is not operational.” Thanks, Samba, and thanks, Windows, for your immensely useful error messages.

Very, very discouraging and even a bit disturbing.

Error when applying group policies on a Samba 4 AD member

Today I ran into the following issue:

C:\WINDOWS\system32>gpupdate /force
Updating policy...

Computer policy could not be updated successfully. The following errors were encountered:

The processing of Group Policy failed. Windows attempted to read the file 
\\YOUR.DOMAIN\sysvol\YOUR.DOMAIN\Policies\{31B2F340-016D-11D2-945F-00C04FB984F9}\gpt.ini
from a domain controller and was not successful. Group Policy settings
may not be applied until this event is resolved. This issue may be transient and
could be caused by one or more of the following:
a) Name Resolution/Network Connectivity to the current domain controller.
b) File Replication Service Latency (a file created on another domain controller
 has not replicated to the current domain controller).
c) The Distributed File System (DFS) client has been disabled.
User Policy update has completed successfully.

To diagnose the failure, review the event log or run GPRESULT /H GPReport.html 
from the command line to access information about Group Policy results.

The solution is simple:

samba-tool ntacl sysvolreset

Found in a mailing list post.

mFI mPower basic use without cloud and controller

Updated 29 December 2014: With the latest software (currently 2.1.4) there is actually a decent, password-protected standalone web UI. I’d recommend you to factory reset the device, set it up from scratch, and set a new username and password from the web UI. You’ll still be able to log in over SSH and telnet, and while I no longer need to access the device directly, I’m sure most of the article below applies.

To upgrade from 1.x series software, which is what I had, you should use scp to upload the new firmware to /tmp/fwupdate.bin. To upgrade from 2.x series software, which has the nicer web UI, just use the web UI. Details.

Just to note: Of course, while I don’t need to use connectivity over terminal, this seems to be used by software such as this nice Android app. The app seems thirdparty (despite the ID being set to com.ubnt.mpower), so it would have been harder to put together if there was no terminal access. Heck, I can even envision management software using not much more than sshfs and ssh to manage a fleet of mPowers (if you happen to need and have such a fleet)…

Original text follows.


After getting the mFI mPower unit, I saw that it really wasn’t planned for standalone use. I was also surprised at seeing no ethernet port; I’m not sure why I thought it’s going to have one.

This is a wifi IP power strip that seems to be designed neither fully for a consumer (why would a consumer need a IP power strip?) nor for an expert. After plugging it in and waiting for it to boot, you’re greeted with a new completely unprotected wifi network. After connecting to it, you’re hijacked in the same way captive portals technologies work. It seems pretty painless to configure a device to connect to a wifi network, and then either to cloud or to a local controller — a chunk of proprietary software that, based on the quick guide booklet, seems to be written in Java. Booklet mentions versions for Windows and OS X, but the website offers download for Linux as well.

I’m however uninterested in having a home machine run 24/7 and waste electricity just to occasionally control a power strip. I opted for the (for obvious reasons less secure) variant of going into the cloud. Unfortunately, the built-in web UI doesn’t give you an option to register nor a hint on doing so. Quick guide does mention the website, which reveals a login panel but no registration.

At least I could configure wifi connectivity without either controller software or cloud — but that seems to be all.

That’s because in October 2013 the service was shut down for new registrations, with promises of coming back. Seeing that was 10 months ago, I began to think I may have purchased a brick.

Luckily, apart from what’s served to the customer on the surface, the device seems to be rather open. I’m unfamiliar with how free and open source it is, but it seems to be built out of relatively understandable components. BusyBox is there, the usual UNIX-like directory structure is there. I also spotted dropbear, which means aside from a telnet daemon, it’s also providing an SSH service.

Default username and password set is ubnt/ubnt. Ouch. First obstacle: How do we change that?

We can use vi to edit /tmp/system.cfg. There it is! Username and password. But wait — what kind of a password hash is that?

Turns out it’s the output of crypt(3). This gets used to generate /etc/passwd.

PHP has the crypt() function as well. PHP’s numerous flaws are irrelevant for such simple use case, so we’ll be forgiven for using:

php <<< '< ?php echo crypt("my_password", "SL");'

where “SL” is the salt. (In the stock password, it was “KQ”.)

You can add new users as well (although I’d highly advise changing at least the password of the default user), like so:

users.1.name=ubnt
users.1.password=KQiBBQ7dx8sx2
users.1.status=enabled
users.2.name=ivucica
users.2.password=AEPbWtbh7XaS.   
users.2.status=enabled

That’s really nice and flexible. But they could have either documented all this (and in an obvious place), or created a web UI (of course, while letting us deal directly through telnet and ssh, too).

To save these settings, punch in save. (Alternative command seems to be cfgmtd -f /tmp/system.cfg -w.) To give the system a chance to apply the settings, reboot.

While at it, you may want to disable the default unprotected wifi network, which for me was numbered 2:

wireless.2.status=disabled

What I also like in this device is that it seems to have the Linux-friendly Atheros chipset in it.

So next. How do we actually read stats or switch an outlet on or off?

cd /proc/power
# enable outlets we want to read stats from or that we want to control
for i in $(seq 1 3) ; do
  echo 1 > enabled${i}
done
# get current power usage
for i in $(seq 1 3) ; do
  echo "active_pwr$i: ${i}"
done
# turn off and on a slot
echo 0 > relay1
sleep 1
echo 1 > relay1

Other functionality is demonstrated and explained by forum member Sequim.

  • active_pwr – power factor corrected power demand
  • v_rms – RMS voltage – zero if outlet is off
  • i_rms – RMS current, as currently delivered
  • pf – power factor
  • energy_sum – totalized energy in Watt-hours delivered via this outlet, probably since last boot

And the /proc/led directory contains some nice controls for the LED.

Really lovely design. It’d have been even nicer if it had been properly documented and if it had a proper web UI shipped in case you don’t feel like dealing with all the power that these controls exposed as a filesystem provide.

Proxying Camlistore through nginx

I’ve went through this in order to secure access to Camlistore and delegate authorization to nginx. This doesn’t help, as I don’t yet have a securely stored password set up to protect it, but the first steps are there. Moving to something like LDAP-backed authorization for authentication on my machine and authentication of Camlistore is probably the way to go.

Without further ado, here is my partial nginx configuration, which should be useful if you already have a domain set up and you’d just like to direct toward Camlistore. I didn’t want to set up a subdomain, as I would have to get a new SSL certificate.

I’m bound to have made a mistake in configuration, so comments are welcome.

server {
        server_name ivan.vucica.net;
        access_log /var/log/nginx/ivan.vucica.net_access.log;
        error_log /var/log/nginx/ivan.vucica.net_error.log;
        root /somewhere/on/my/disk;
        listen 80;
        # skipped...

        location /camli {
                return 302 https://ivan.vucica.net:3180$request_uri;
        }
}
server {
        server_name ivan.vucica.net;
        access_log /var/log/nginx/ivan.vucica.net_access.log;
        error_log /var/log/nginx/ivan.vucica.net_error.log;
        root /somewhere/on/my/disk;
        listen 443 ssl;
        # skipped...

        location /camli {
                return 302 $scheme://ivan.vucica.net:3180$request_uri;
        }

        ssl_certificate /ssl/directory/on/my/disk/startssl-vucica.net.chained.crt;
        ssl_certificate_key /ssl/directory/on/my/disk/startssl-vucica.net.key;
        # ssl_protocols       SSLv3 TLSv1 TLSv1.1 TLSv1.2;
        # ssl_ciphers         HIGH:!aNULL:!MD5;

        # from: https://community.qualys.com/blogs/securitylabs/2013/08/05/configuring-apache-nginx-and-openssl-for-forward-secrecy#comment-3794
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_prefer_server_ciphers on;
        ssl_ciphers "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS";

        # for ssl cache - see http://nginx.org/en/docs/http/configuring_https_servers.html 
        keepalive_timeout   70;
}
server {
        server_name ivan.vucica.net;
        access_log /var/log/nginx/ivan.vucica.net_camli_access.log;
        error_log /var/log/nginx/ivan.vucica.net_camli_error.log;
        root /somewhere/on/my/disk;
        listen 3180 ssl;

        location /camli {
                rewrite ^/camli/(.*) /$1 redirect;
                rewrite ^/camli$ / redirect;
        }
        location / {
                proxy_pass http://127.0.0.1:3179;
                proxy_connect_timeout 5;

                # not using after all, as it would need access to /etc/shadow.
                # see http://web.iti.upv.es/~sto/nginx/ngx_http_auth_pam_module-1.3/README.html
                # auth_pam "Secured Camli";
                # auth_pam_service_name "nginx";

                auth_basic "Secured Camli";
                auth_basic_user_file /path/to/ivucica-camli-user_file;
        }

        ssl_certificate /ssl/directory/on/my/disk/startssl-vucica.net.chained.crt;
        ssl_certificate_key /ssl/directory/on/my/disk/startssl-vucica.net.key;
        # ssl_protocols       SSLv3 TLSv1 TLSv1.1 TLSv1.2;
        # ssl_ciphers         HIGH:!aNULL:!MD5;

        # from: https://community.qualys.com/blogs/securitylabs/2013/08/05/configuring-apache-nginx-and-openssl-for-forward-secrecy#comment-3794
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_prefer_server_ciphers on;
        ssl_ciphers "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS";

        # for ssl cache - see http://nginx.org/en/docs/http/configuring_https_servers.html 
        keepalive_timeout   70;
}

Camlistore itself is configured to listen only on 127.0.0.1. It doesn’t handle authentication, as using localhost authentication would require running nginx and camlistored under the same user. Adding a username+password authentication and using internal SSL would limit my long-term options for configuring authentication.

{
    "auth": "none",
    "listen": "localhost:3179",
    "identity": "AC5742DD",
    "identitySecretRing": "/path/to/camlistore/identity-secring.gpg",
    "blobPath": "/path/to/camlistore/blobs",
    "sqlite": "/path/to/camlistore/camli-index.db",
    "baseURL": "https://ivan.vucica.net:3180/",

    "shareHandler": true

}

To generate the .htpasswd file, refer to nginx documentation. Here follows an example; consider hard whether this is secure enough and appropriate for you.

printf "John:$(openssl passwd -crypt V3Ry)\n" >> .htpasswd # this example uses crypt encryption

Don’t forget that this will store the line in your .bash_history. (One way to avoid this specific issue is to prefix the command line with a space. Think hard whether this is enough for you.)

Mails appearing from d1stkfactory

If you have a Debian machine and emails appear to be coming from hostname @d1stkfactory, edit /etc/mailname and set this to your actual FQDN. Another instance where I found it locally is in /etc/exim4/update-exim4.conf.conf, which is almost certainly not read by my installation of Postfix (which does explicitly refer to /etc/mailname in one place).

This appeared on a machine running on DigitalOcean, so I presume “d” refers to DigitalOcean, and “factory” refers to their internal disk image building service. That’s just speculation though.