Ep 107: Idempotify!

Functional Design in Clojure

Each week, we discuss a different topic about Clojure and functional programming.

If you have a question or topic you'd like us to discuss, tweet @clojuredesign, send an email to [e-mail], or join the #clojuredesign-podcast channel on the Clojurians Slack.

This week, the topic is: "handling endless errors". We discover when giving up is the way to get ahead.

Our discussion includes:

• Finishing up our Sportify tracer-bullet implementation.
• What context can you put in a file name?
• Tricks on working with time information.
• Trying to "fix" a situation vs adding more context.
• What is a "positive representation" of a problem?
• What are "fundamental" versus "derived" facts?
• What information tends to be more stable?
• What information should you try to co-locate?
• How do you handle resources that are not currently available?
  • How many times should you retry?
  • How long should you wait between retries?
  • How do you figure that out?
• What is idempotency? How can it help?
• At what point should you stop trying to handle errors?
• Can you have too much automation?
• How can you troubleshoot intermittent problems?

Selected quotes

• We don't need a time zone offset because we know it's in UTC!

• In the spirit of building up the language to meet our domain, we can write a pure function!

• We want a deterministic way to go from this kind of common information into all the other bits of derived information.

• I was really hoping that we would finally be done with the errors and we could just get a highlight clip, but yet again, the world has conspired against us to make our life difficult as a programmer!

• Hold on! Hold on! The first thing we should do is run our process again because maybe the error will just go away!

• The problem does not go away in this instance. The problem just goes back to being hidden!

• The frustrating thing about programming is that code will do exactly what you told it to do.

• Clojure is already positive about nothing, that's what we call nil, so why not be positive about bad stuff too?

• Now I'm personifying the function as myself!

• It's better to reference information by its main identifier as opposed to some derived identifier.

• The context is all over the place: some context in memory, some context in the file system, and some intermediate context in the imperative function.

• That's the last problem we encounter, right? You never know what the world's going to throw at you!

• Why don't we just run it again? Let's just run it again! Maybe it'll be ready now... Maybe now... Maybe it will be ready now...

• And then we hit another error which is: the MAM rejects us for making too many requests!

• How long do you wait? Should you back off? That adds a lot of complexity to the code at that level.

• Because adding idempotency increases the complexity of that part of the solution, we only want to add it where it's necessary.

• The error condition is happening right now, so let's write the code to fix it right now.

• That's the way automation is at some point in time: a human needs to do it.

• You can find yourself in a situation where you're trying to do too many automatic things. Is it really worth doing? You cannot solve every possible permutation once you're interacting with the real world.

• It's okay to put up the guardrails, and if I'm outside them, robot me throws up my hands and says, "Human please!"

• But what happens when you run it again and the clip is ready? Now your retry logic doesn't get tested.

Links

• Sportify! series:
  • Ep 101: Sportify!
  • Ep 102: REPLify!
  • Ep 103: Explorify!
  • Ep 104: Assembleify!
  • Ep 105: Codify!
  • Ep 106: Robustify!
• Related episodes:
  • Ep 027: Collected Context
  • Ep 029: Problem Unknown: Log Lines