marcosilva_eu

How (not) to clone a linux installation

2021-05-17T00:00:00Z

Recently I decided to move my second Linux Mint environment to another disk. One tiny detail left me stranded in GRUB console.

I keep a second Mint installation in a portable hard drive (actually a very small thing I can plug into USB in any hardware) that I don't use for anything serious, some browsing, Netflix and the sort, leaving everything development-wise to my beefier main system.

Not long ago, I got a small laptop that was perfect for this form factor, tempted me to let go of the USB thingie altogether and just use that laptop like a normal person. But, not wanting to reconfigure Mint, I decided to simply dd the entire thing to the new disk.

This was uneventful. dd did its job, I rebooted the system, booted it once from USB to check I didn't break anything, rebooted and let it boot from internal storage. This seemed ok so I unplugged the USB drive and carried on with the important task of watching some The Crown.

A couple of days later, a routing Mint update in this system exploded GRUB and the system stopped booting. Having recently used the USB disk with the original installation in other hardware and updated without incident, I tried booting from external, was cheerfully welcomed by GRUB, selected the first entry and was greeted with

Loading Linux linux ...
error: file `/boot/vmlinuz-5.4.0-72-generic' not found.
Loading initial ramdisk ...
error: you need to load the kernel first.

"Well, that can't be right" and sure enough, the external USB drive booted fine in another laptop. So the issue had to be in the system itself. Rebooting from external, I press e to check what GRUB was dreaming of but everything seemed fine, I compared the root UUID with the external in another system and it was a match. I'm sort of kinda familiar with GRUB's console, so a Ctrl+C later I was staring at a grub> prompt. So, let's start with the basics:

> ls
(proc) (hd0) (hd0,msdos1) (hd1) (hd1,msdos1)

Expected until now, since I booted from external hd0 would be my external (working) system and hd1 would be the broken internal system

> ls (hd0,msdos1)/boot
(a bunch of vmlinuz, initrd, etc, including the vmlinuz supposedly MIA)

> ls (hd1,msdos1)/boot
(another bunch of vmlinuz, initrd, but not the MIA vmlinuz)

Well, this also made sense, as the internal installation broke before the last update (and last kernel) I got in the external system, so the next step was to manually boot linux from GRUB:

> set root=(hd0,msdos1)
> set prefix=(hd0,msdos1)/boot/grub
> linux /boot/vmlinuz-5.4.0-72-generic
> initrd /boot/initrd.img-5.4.0-72-generic
> boot

Finally I start seeing Mint booting but soon enough I get the dreaded

Target filesystem doesn't have /sbin/init. No init found. Try passing init= bootarg.

and I get dropped into the emergency console. This one confused me a bit more. After fsck -f the disks and retrying the reboot, I still got the same result. And that's more or less when it hit me: I did a very dumb thing when cloning this Mint installation.

To confirm this, from the emergency console I ran

> blkid
/dev/sda1: UUID="xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx" TYPE="ext4"
/dev/sdb1: UUID="xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx" TYPE="ext4"

and obviously the UUIDs matched, as I totally forgot to change the UUID of the target disk after running dd. So GRUB was dutifully booting from external, but when searching for the root to load the kernel, kept trying the internal disk as it was the first block device with the root UUID.

I mounted both disks from the emergency console, checked fstab and yep, I totally forgot about changing the UUIDs. I didn't have uuidgen in this console so I wasn't too keen on messing with GRUB and fstab from here. However, GRUB lets us easily bypass this sort of stupidity from its console. So, another reboot from external, and now my manual boot sequence was

> set root=(hd0,msdos1)
> set prefix=(hd0,msdos1)/boot/grub
> linux /boot/vmlinuz-5.4.0-72-generic root=/dev/sda1
> initrd /boot/initrd.img-5.4.0-72-generic
> boot

after a while, I was greeted with Mint's login screen. Now to fix this screw-up: open a terminal

> sudo tune2fs /dev/sdb1 -U `uuidgen`

Confirmation prompt and a while after (changing the UUID in a live disk takes a bit) I double checked

> blkid
/dev/sda1: UUID="xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx" TYPE="ext4"
/dev/sdb1: UUID="yyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyy" TYPE="ext4"

Mounted the internal system disk and replaced the new UUID in /etc/fstab.

Another reboot from external, select first entry at GRUB screen, and the system booted fine.

Next I tried recovering booting from the internal disk — and hit a roadblock. GRUB displayed but as I hadn't run update-grub it would be looking for the external block device which I had unplugged at this point. So trying the same routine I got to a different issue

> set root=(hd0,msdos1)
> set prefix=(hd0,msdos1)/boot/grub
> linux /boot/vmlinuz-5.4.0-66-generic root=/dev/sdb1
symbol grub_calloc not found

GRUB was broken on internal. Booting back from external I tried a fresh GRUB install to that disk

sudo mount /dev/sdb1 /mnt
sudo grub-install --root-directory=/mnt /dev/sdb

which after another reboot no longer complains about grub_calloc but unfortunately also hang when loading the kernel without any more info.

However, this one will be fixed next time. For now, my second system is working again and I can drown my sorrows in another episode of The Crown (I know, I'm ridiculously late!)

Update: I actually figured out the kernel hang: I'm missing a kernel module needed to drive the internal disk. Fixing this one will be uneventful, so it doesn't merit a separate blog post. What a downer :)

On the importance of design and how it affects everyday life

2014-12-05T00:00:00Z

Welcome to the second entry of my commentaries on The Design of Everyday Things by Don Norman and how it applies to the everyday software design. In this entry we'll focus on the interaction of the user with complex software and what the natural approach would be to design this kind of system. For this it is important to start with the definition of knowledge in the mind (or brain) vs. knowledge in the world (or in the system).

Knowledge in the brain is essentially all the knowledge that we must know before hand when executing an action. There are no cues to guide us through the execution of said action, only our knowledge of how to perform it. This is still prevalent in business or industrial software. Knowledge in the world, on the other hand, means that the information we require to execute an action resides in the physical world itself, or in our case, in the system itself. We can easily execute this action without familiarity with it or without memorizing a set of steps, because all needed information is available to us when performing the action, embedded in the object or system being used.

When people fail to follow these bizarre, secret rules, and the machine does the wrong thing, its operators are blamed for not understanding the machine, for not following its rigid specifications. With everyday objects, the result is frustration. With complex devices and commercial and industrial processes, the resulting difficulties can lead to accidents, injuries, and even deaths. It is time to reverse the situation: to cast the blame upon the machines and their design. It is the machine and its design that are at fault. It is the duty of machines and those who design them to understand people. It is not our duty to understand the arbitrary, meaningless dictates of machines.

Having developed business software for most of my career, it's worrying that this simple truth is still largely ignored. Such complex applications warrant the use of the best possible interaction paradigms, as well as innovative design which is familiar to users from everyday apps, giving the user less data to learn (again). I've heard more than once the old excuse that "this application is very complex, so we'll just explain everything to the users during training, don't worry so much about the design" or "it's already complicated, they'll just learn this new way of doing things". Knowledge in the mind is notoriously unreliable and without great design care and as much knowledge in the world (or system in this case) mistakes will inevitably happen, which is of paramount importance in systems in which undo is not easily done.

Unless it is your ambition to become a nightclub performer and amaze people with great skills of memory, here is a simpler way to dramatically enhance both memory and accuracy: write things down. Writing is a powerful technology: why not use it? Use a pad of paper, or the back of your hand. Write it or type it. Use a phone or a computer. Dictate it. This is what technology is for. The unaided mind is surprisingly limited. It is things that make us smart. Take advantage of them.

In software, this kind of action can be included in the system design itself. Instead of an endless list of fields the user must fill, provide the relevant information in each screen. This will, in general, increase the number of screens in your system, but will allow for less errors and less effort from the user, due to the minor reliance on knowledge in the brain. If this is not possible or feasible, allowing for notes or comments from the user to be updated during the execution of a complex action may be sufficient. Let the user type in notes that give him the necessary context to perform the action without errors. This contemplates information not available in the system as well. There is an exception to this rule, and that is the use of cultural knowledge. This knowledge is present in the subconscious of the user, as it is something ingrained into their everyday lives. However, it is easy to cause grave mishaps when relying on this type of knowledge: it differs from where the user is from. In this case and if you decide to rely on cultural knowledge, beware of your target audience and design accordingly, changing cultural references and parameters when such is needed.

As architects and designers, it is our mission to provide reliable software that simplifies the user's everyday life. In the corporate world, we, almost exclusively, build tools to ease common tasks. If from the start we focus on the psychology of ours users instead of the data structures we are working with, we'll build better products that people love to use and make everyone's lives a little bit better.

Don Norman's Error credo

2014-10-06T00:00:00Z

I have been reading Don Norman's seminal The Design of Everyday things and thought to share some passages that resounded during my reading. I'll be posting a few of these in the near future with compilations of small chunks of text. For now, I leave you with the first of those quotes, one that is of great importance when designing tools for everyday use.

True collaboration requires each party to make some effort to accommodate and understand the other. When we collaborate with machines, it is people who must do all the accommodation. Why shouldn’t the machine be more friendly? The machine should accept normal human behavior, but just as people often subconsciously assess the accuracy of things being said, machines should judge the quality of information given it, in this case to help its operators avoid grievous errors because of simple slips. Today, we insist that people perform abnormally, to adapt themselves to the peculiar demands of machines, which includes always giving precise, accurate information. Humans are particularly bad at this, yet when they fail to meet the arbitrary, inhuman requirements of machines, we call it human error. No, it is design error.

Don Norman

Easy REST API mock-up testing

2013-12-04T00:00:00Z

One of the things I've recently been working on is a large HTML5 website for a client. This site communicates with a WCF-based REST backend, using a Knockout plugin for the calls that, in the end, basically uses the usual $.ajax call from jQuery.

As I started the development of the proof of concept, I came to a point in which I wanted to display some dummy data in the UI without hard-coding into the source, as some fellow developers were to take over this codebase and write the actual production website. What I needed was a fast way to deploy some mock API calls and responses. If you deploy automated testing using a framework like Selenium WebDriver, which I also used for this project, this can be very useful for UI testing.

I was deploying this website using Internet Information Services. IIS has a very nice feature, on which you can pretty much configure all of the website using web.config files. This has de advantage that if you include this in a Team Foundation Server build which automatically deploys the website, then shipping these web.config files means wherever you run it, it will always be correctly configured. Also, web.config is accepted not only at the root of the website but anywhere.

Now, my API has a URI format of

http://example.local/<service_group>/<service>[query param...]

If you make the root of the API configurable, which is always a good idea anyway so you can switch between e.g. productive and testing servers, you can easily point this to another server with a similar API structure. So, the solution to my problem was easy!

I created an api/ folder in my proof of concept and placed a web.config file there specifying that all the files in that folder and respective children should have the application/json MIME type. Then I simply create a service group folder and a file named service in it, and in this file I place the mock reply. If the URI takes parameters, e.g. a GET, I keep creating folders and the last part of the URL is the file. Easy.

Now, if you try to do this from the IIS management console, it will ask you for an extension. Of course you can say that the extension is '*' to match all files, but this will only match files with extensions, which was not what I want. A workaround is to specify an extension of '.' without the quotes. The important part is that dot. It tells IIS that the files with no extension need to be served with application/json MIME type.

Take into account that if you are serving the API files from the same website, you won't be testing against cross origin requests, so if when changing to the real API server no call goes through, remember this.

And you're done! Pointing your AJAX calls to the local URL will get you the mock replies. While you're at it, stick another web.config file into the root to disable caching. Saves you a ton of time debugging!

Checking in changes in a different branch in TFS

2013-06-06T00:00:00Z

Recently I had the need to check in some changes in a different branch. In git this is very easy, but as far as I know in TFS not so much.

Using the approach detailed by Torbjörn Bergstedt at Stack Overflow, I created a shelveset on my source branch and subsequently migrated said shelveset onto the target branch

tfpt unshelve client /migrate /source:$/MY-PROJ/SourceCode/dev-branch /target:$/MY-PROJ/SourceCode/UI-branch

to which tfpt helpfully replied

Creating the backup shelveset which will serve as a snapshot of your workspace's current state...

asking me then to deal with the merges. As it was a new branch, selecting auto-merge solved all of the conflicts. At this point, I got

The backup snapshot was preserved as the shelveset myshelve_backup1.

Analyzing this shelveset, however, turned out to be a bit weird. My source shelveset had about 8 changes. The target changeset (and as I asked for an unshelve, also my workspace) has about 360 changes. Why?

I have a folder at the same level of the source and target branch. This is a regular folder, not a branch in itself. For some reason, even though I explicitly requested a migration for the source to target branch, this top level folder was included into the target shelveset as with every item added, which is in itself weird: the files obviously already exist.

That being said, I undid my pending changes in that folder and continued to work on the target branch. So, this works, but I wouldn't recommend its use on a regular basis.

If you know any better way to do this, please, find me on Twitter and let me know.

Mocking an internal method with generic parameters using jMockit

2013-01-07T00:00:00Z

Last week, in order to write a test in a way that assumed less about the internal class, I got to the point where the best approach would be to mock an internal method, responsible for loading some metadata. I could test this from the behaviour of the class with my passed inputs, but for this I would have to know what logic resides inside it. This was, however, no good here as this was not the logic to be tested.

Our scenario

We use jMockit as our mocking framework in the project and almost exclusively with the Expectations API. This API provides an invoke method, which can, in turn be used to mock internal methods. Logically, I followed this route, but hit a (major) snag right after it.

My method has a simple but, for this topic, tricky signature

private void loadMetadata(String id, Map<String, String> arguments)

What invoke does is that, through Reflection at runtime, it will determine the type of the parameters and, together with the method name, it will determine what is the best match method in the class. There is, however, one basic flaw to this approach, which some background on Java generics will point out

Java Generics: not like .NET

In .NET, generics are first class citizens and an integral part of .NET 2.0+. They are compiled to bytecode and are understood by the CLI. Java, however, uses generics more as an afterthought, in that they are not understood by the JVM. When compiling a Java application, all generic classes will suffer what is called erasure. What this means is that they will be translated to their bounds, or at its limit, to Object. The compiler does a bunch of other stuff like casting to maintain consistency, but the point is that the bytecode contains nothing more than ordinary classes. A simple run with invoke will show just how naive this approach is

Invoking our way through

Using jMockit's Expectations API, on can declare in the Expectations block

invoke(obj, "loadMetadata", "id", args); times=1;

which means that the loadMetadata method on object obj with "id" and args as arguments. As args is a generic Map, which is an interface, the object actually passed is an HashMap<String, String>. So what happens when you run the test?

java.lang.IllegalArgumentException: No compatible method found: loadMetadata(java.lang.String, java.util.HashMap)

So what happened here? Quite simply, the generics were erased, from HashMap<String,String> into HashMap at runtime. This will not matched our declared signature as stated above. So, we're out of luck. Or are we?

Why jMockit pays

One thing I had to endure from my team colleagues for a while was why did I choose jMockit as a mocking framework when it is harder to use than most. And I always present the same reason: flexibility!

jMockit packs not one but three APIs for testing, Expectations, Verifications and Annotations. Until now I had no reason to use Annotations (noted as to being the most powerful of the three), but some testing actually proved that this is easily done with it. Annotations is, however, more verbose and harder to read for someone not familiar with it.

Annotations is, from the start, a partial mocking API. This means that when you are mocking any class, you specify only what methods you want to mock and the remaining will remain as they are in the original implementation. Also, taking advantage that in jMockit you mock not instances but entire classes, you can easily mock a specific method in all instances of a given class straight in the JVM (how cool is that?) This does not work in the same way as Expectations' invoke. It will replace the original method signature and content with your code. So, how do we do it?

//jMockit annotations API! woohoo!
 new MockUp<MyClass>() {
 public String magic; //for testing, no getter/setter here
 @Mock
 void loadMetadata(String id, Map<String, String> arguments) {
 assert id.equals(magic);
 }
 }.magic = magic;

We start by declaring a new mock for the desired class and writing our methods. It's important to note that I dropped the private from the method declaration. This is necessary. If you do not, jMockit will error out. This does not, however, change the method's visibility at runtime. Also important is to annotate the method as a Mock. As I need an external string from this object, I declare a public variable and fill it directly when declaring the anonymous class (it can be argued that this variable should be private and a public getter/setter should be used, but for test purposes this is enough.)

And that's it! jMockit will replace the loadMetadata method in every object of MyClass right in the JVM. Running the test with a breakpoint on the assert will show our mock is being triggered as expected.

So this is how you can easily mock an internal method with generic arguments. I'm in no way a jMockit expert, so if you find a better approach to do this, please sound of in the comments.

Bye!

Sharing Team Foundation Server workspaces

2012-11-08T00:00:00Z

Recently I needed to allow another user to use one of my TFS workspaces. This user needed to be able to check-out a file, change it and check it in again. This turned out to be quite harder than, in my opinion, it ever should. First, what Microsoft says

Public workspaces

According to Microsoft, in TFS 2010 you can now share workspaces. When editing or creating a workspace, you should see a combo

which you can use to change this setting. The set permissions are

Private workspace - only the owner can use the workspace
Public workspace (limited) - all other valid team members have the Read and Use permissions
Public workspace - all other valid team members have all the Read, Use, CheckIn and Administer permissions

You cannot finetune the permissions either via the GUI or the tf command. It's however possible to change them using the TFS API.

Seemed like setting the workspace as non-limited public should work, but I faced another problem. The other user would be able to use the workspace and checkout the files, but it would not be able to check them back in again

TF14098: Access Denied: User DOMAIN\\OtherUser needs Checkin, CheckinOther permission(s) for $/somepath/somefile.txt.

This one was trickier to resolve because nobody in my company had ever used the CheckinOther permission for any project and such a permission did not appear in the project security tab. After a little bit of googling, though, I found out that these permissions actually reside in the folder security settings. So I added the other user to the Builders group, went to the folder which I needed the other user to be able to use, right-click, Security, selected the group and gave it the Check in other users' changes permission (this has to be done by an administrator account)

Click ok and you are good to go!

Using Java Mail API with untrusted certificates

2012-11-06T00:00:00Z

Today I was playing around with the Java Mail API using channel security and hit a snag. Our SMTP server supports no security and TLS. Everything was great using no security, but using TLS we get

javax.mail.MessagingException: Can't send command to SMTP host;

nested exception is:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The problem here was the SMTP server certificate: it was untrusted. This will happen every time with self-signed certificates. In my case, the certificate was signed by an untrusted certificate authority (the CA certificate itself was self-signed.)

I had two problems: I need to trust the server certificate, but more difficult than that was that I needed to actually obtain the certificate because I didn’t have it.

Problem one: obtaining the SMTP server certificate

The easiest way to obtain these certificates is to use openssl’s s_client. s_client is provided with openssl to debug SSL/TLS servers. You are able to observe the entire protocol execution and obtain most of the information you need to resolve whatever problem you’re facing. In my case, for the SMTP + TLS scenario, I just ran

openssl s_client -connect mail.example.com:25 -starttls smtp > smtp.txt

This will take a long time to complete and will seem stuck, but you can cancel it soon after the beginning of the execution as you’ll already have the certificate.

For the SMTP + SSL scenario, I ran

openssl s_client -connect mail.example.com:465 > smtp.txt

This actually seemed stuck earlier than the TLS command, but like that one you get the certificate right in the beginning. Feel free to cancel it.

Open the smtp.txt file. You will see a certificate chain followed by Server certificate. Copy the bit starting with BEGIN CERTIFICATE and END CERTIFICATE into a smtp.crt file. Opening this file you should be able to check the certificate properties and its respective thumbprint.

Congratulations, you now have the server certificate. Now you just need to trust it.

Problem two: trusting the certificate

There are two options for trusting this certificate: you can add it to the cacerts keystore in your JVM so that all java apps can trust it or you can add it to a separate keystore and specify in your code to use this keystore.

I followed the first approach as this certificate was actually ok to be trusted. So to trust this certificate, open a terminal window, go to <jre_home>/lib/security and type

keytool -import -keystore cacerts -alias <alias> -file smtp.crt

No need to specify alias, but the default is mykey which made it nearly certain I wouldn’t remember what certificate this was supposed to be. It can be anything, though.

And you’re done. Remember, you need to execute the keytool import into the cacerts for the target JVM!

Creating a patch in WinMerge

2012-09-20T00:00:00Z

Today a colleague was a bit puzzled by how I generated a patch in WinMerge. This can be useful if you're not using something like Git in your workflow, or if, like me, Git just isn't working for you (in my case, a unix file being edited in windows resulted in a patch that replaced the entire source file).

So, open both the versions of the source file in WinMerge, the one you want to be the target version to patch on the left and the current one on the right. You can do this manually or in a number of ways in Git, but in my case I used

> git difftool HEAD~3 HEAD

to produce a patch that embodies my last three commits. Note that to use difftool, you need to have Git configured correctly to open WinMerge. You will see the base version on the left side and the current one on the right, just like we want it. Then, simply go into the Tools menu and click Generate Patch. A window will appear with a bunch of options. I mainly use the default, but you have three different syntaxes for the patch files, as well as options regarding context and some less interesting stuff. Browse to/type your target file for the patch and confirm the window.

Voilà, a very simple way to generate patches using WinMerge.