Again, my question was closed. It was, as far as I can tell, focused. I even edited it to narrow the scope. There isn't any question there's an answer to it. How can I improve?
Cheap and cheerful way to do leader election on AWS
Any of the following answers would have been perfectly good (I don't know any of these answers, and Google was very light in providing these):
- "You can use DynamoDB as an atomic store. It is super easy to setup x, y, and z and you can use it as a leader elector (using the text description you said earlier)"
- "You can use S3 as a store, as it provides atomic and versioned writes. Here's how you write to the S3 bucket using versions, and here's how you would query the results to get the leader."
- "Actually, AWS Secret Manager is a great place to do leader election, as it has strongly consistent writes, and prevents overwriting."
There's nothing about a software architect or other complex items; it's just that feature (strongly consistent storage without any overwrites) that would solve it.
I understand that I could have specified further on not using Java, but my goal was not to use (as much as reasonably possible) any custom code, and just reuse an existing solution (e.g., Dynamo, S3, etc. which have many millions of person hours solving leader election).
I do want a programming solution. E.g., imagine the following (which is how I'm currently thinking about it).
Possible (great) answer:
What you could do would be to use S3 versioning as a strongly consistent way to write. For example, here's how you would do that in Python (all pseudocode, obviously):
S3BUCKET = "lock-bucket"
LOCKFILE = "lock-file"
def writeLock():
s3.write(getHostName())
def writePeeringString(peeringString):
s3.write(peerString)
def getAllFileVersions() -> List:
return s3.getAllFiles(S3BUCKET, LOCKFILE)
def isThisNodeLeader() -> bool:
# Gets all files as a list, ordered by version
firstFile = s3.getFileVersion(S3BUCKET, LOCKFILE, 0)
return getHostName() == firstFile.contents()
def main():
writeLock()
if isThisNodeLeader():
peeringString = startServer()
writePeeringString(peeringString)
else:
while:
sleep(5)
for f in getAllFileVersions():
if isHostName(f.contents()):
pass
else:
# The file contains a peering string, use that to start the server and then break
startServer(f.contents())
break
There are many such solutions. They require a minimum of coding, but absolutely require coding. It does not attempt to do the very complicated work of leader election; it basically relies on the S3 Versioning API to accomplish this.
I tried to ask this. Surely someone has run into a problem (and proposed a solution) similar to this.