How can we keep the uid and gid constant

[michaelwittig]

We do not guarantee constant uid and gid for created local users at the moment. How could we handle this?

[uberbrady]

Brain dump. I think that Proposal 4 has the most 'legs' maybe. But I would keep pondering for a little bit:

---

Well, let's look at what we do have in IAM that's constant. Is it just the ARN? E.g. something like:

arn:aws:iam::A_BUNCH_OF_INTEGERS:user/my_username

If so, that's a good place to start to try and 'generate' our userid?

Proposal 1:

So I'm thinking something, like, we take the CRC32 of that thing, modulo 'x' it (for some value of 'x' TBD), add an offset to make it not conflict with the built-in /etc/passwd users (probably an offset of 1000 would do?).

Problems:

• CRC32 isn't collision-free.
• Can your ARN change? I don't think so; it seems pretty fixed.
• Even if CRC32 were collision-free, anything modulo'ed low enough could definitely collide. I think the thing you'd have to modulo against would be, like (65535-1000) - totally collision-town.

Proposal 2:

Well, if we look just at the username - my_username - and we assume that it's only a-z0-9, so that's base36... shit, do we still have to fit it in 2^16? I think you can only end up squeezing one more letter.

Proposal 3:

If there were a consistent way to list IAM users - like, they always show up in the same order - and if they somehow showed a gap for deleted users - we could assign uid's in order? I think the biggest issue here is deleted users; for old servers it would be fine because they would 'know' about the missing user. For new servers, who never knew about the gap, they would try and misnumber

Proposal 4:

This one is well and truly horrible, but it might be the best bet.

We attach a magical, no-permissions, valueless role or permission to each IAM user. Like, maybe an inline policy. Maybe the policy is completely blank. But the name of the policy is something like:

aws-ec2-ssh-uid-1003

And from there, all servers now know that that IAM user is always to be uid-1003? (And similar with gid?)

One note - we'd have to keep track of "last_used_uid" somewhere. Hell, maybe another IAM user, who exists just to be a counter (also has no perms?) - aws-ec2-ssh-last-uid

[michaelwittig]

I also though about hashing the username (or ARN, doesn't matter in this case) but I'm not aware of a mechanism that does not come with collisions as you mentioned.

So I think we need to keep some state somewhere like Proposal 4 suggest. But instead of the IAM user I was thinking of a DynamoDB table or something. There we could easily keep track of users and their assigned uid / gid. But it makes the whole thing more complicated to setup...

[mvanbaak]

I have also been thinking about this. There is nothing really usable in IAM. hashing the names will be a problem, as it's too easy to have collisions.

The best thing would be tags, but AWS did not implement tags for IAM users :(

So I could only think of:

• iam group per user like proposal 4 above
• use some service to store the mapping. Either a mapping on S3, Dynamo, or, in my opinion, better yet simpledb

All of this sounds like a lot of extra complexity, and if we go implement this I would strongly opt for making it optional, disabled by default.
Specially since this has, in my opinion, a limited usecase (efs, running a service as a user on more then one instance where the service depends on uids)

I only had to handle this once (our jenkins machines use efs for storage. I ended up picking a uid for the jenkins user and put this uid in the userdata, and only fix the uid for the jenkins user)

Maybe we can open a feature request with AWS to allow tags on IAM users? ;-)

[michaelwittig]

I'm a fan of keeping things simple. This project started as a few lines of bash :)
Unstable uis and gids are documented as a limitation. My opinion: It okay to keep it like this.

[mvanbaak]

I agree :)

[uberbrady]

That's where I'm leaning currently too. But it's still something we should think about; consistent uid/gid's would enable things like NFS/EFS, and there are some people out there who might think that's a big deal.

I did an experiment with a simple 'inline policy' -

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt1494299234000",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeAvailabilityZones"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

(that just allows you to list ec2 availability zones, but any simple, near-empty or meaningless policy would work).

I then attached this inline to my user using the console.

Finally, I was able to do this:

aws --profile=production iam list-user-policies --user-name brady
{
    "PolicyNames": [
        "fake-uid-1001"
    ]
}

I honestly don't think it's too awful? Hrm?

[michaelwittig]

The only concern I have is that the AWS API is eventual consistent. Which means that you can not use the API to assign unique values.
And it could also be that two EC2 instances are started at the same time (e.g. by an Auto Scaling Group). They will not know each other and hit the AWS API at the same time, This could also lead to messed up uids.

[mvanbaak]

That would only be a problem right after adding this inline policy right?

My main concern is: how do we handle a mix? Some users have this policy, others dont.
We would have to get the users, then sort on the policies, etc. or is there another solution?

It also adds a lot of api calls which means less users/instances can be managed this way.

Tricky feature.

[uberbrady]

Yeah, and to the earliest point, it might not be worth it to do at all. But let's at least keep stepping through it.

(oh, and for people who don't have the policy, when the update-users script fires, it would start assigning them in order).

The eventual consistency issue is definitely still an issue, but only at the moment where you are trying to assign a new uid. Once it's assigned, it's pretty painless on currently-existing servers as well as brand new ones - you just use it to set the uid, that's it. But if you have 'n' servers, and their cronjobs all fire at the same time - chances of some kind of conflict are definitely there. I think some of this is mitigated iff the user-listing we get back from IAM keeps the same order; so long as we keep appending new UID's in order to that, then even if two people try to do it at the same time, it might be OK? Would we still need some kind of 'marker' (fake IAM user?) so that you don't inadvertently re-use the last issued uid, or is that not an issue? E.g. if you have uid 1001, 1002, 1003, and delete 1002, that's easy, next one is 1004. But if, instead, you deleted 1003, wouldn't you end up trying to re-use 1003? Is that bad?

If eventual consistency is an issue, we might be able to use the CreateDate attribute of the IAM user and only try and assign the uid things if the CreateDate is more than 5 minutes ago; that way most of the eventual-consistency has shaken out. (And adjust '5' appropriately, blah blah).

So I still think it's doable, and I think it'd even probably work, but I worry that if we try and go this way, we'll end up having to build a full identity-provider á la LDAP or something, and deal with nssswitch and/or a whole bunch of other craziness. Though, to be fair, we already do interact with PAM, so maybe that's not the worst thing ever....oh, I dunno.

Anyways, just wanted to at least try and think the idea through to its logical conclusion.