What? I have to think about things when writing a simple CRUD API? Isn’t it just a simple API call which makes another DB call?
Many freshly graduated engineers think that way but there are a couple of things which — when you think about — can lead to fewer comments on your PR and overall help you improve your critical thinking.
It’s quite possible that you won’t have to think about all of this if you are building a very simple system but most often when you are working at not-small scale, you will have to think about some of them to an extent.
For the sake of simplicity, we will be talking about a simple CRUD interface. For more complex interfaces, take the same points and apply to your usecase:
- Logging
- Database layer
- Cache Layer
- Idempotency
- Analytics Requirements
Logging
Let’s start with something easy, eh? Your company will most likely will have some logging platform(Corallogix, Datadog, etc) where logs from various servers and services can be stored, queried and monitored.
Logs are not just plain text, they are log objects which contain keys and values which, in turn, help in effective querying and filtering.
A good log statement will contain a key and/or meaningful message which will help an engineer debug issues when they come in production.
Be sure to log the correlation-id as well. The correlation-id links together logs from various services and code for a particular user action.
Database Layer
Almost every API has to interact with the database, it can be Postgres, MongoDB, MySQL, etc.
One of the major points newbies miss is transactions.
If you are interacting with a database, it will most likely have some concept of Database Transactions. If your code fails midway, then you need to handle the error and rollback any DB changes you did up till the point of the error.
If your code doesn’t perform DB actions inside a transaction, it can leave your DB in an inconsistent state which can bring unexpected errors in other systems.
A lot of distributed systems employ async processing where many services listen for changes in a particular DB. This mainly happens through a CDC(Change Data Capture) or Queues so when you do update a table row, think about what how that change will be used by other systems.
Cache Layer
Suppose you are building an API to update a row in the DB.
Think if you need to update the cache too? Do you do it inside a transaction?
To answer the above questions, you need to be familiar with how data(which you are updating) will be accessed(by clients, other services, etc).
If you want to know more about caching strategies at backend, read this blog post ;)
Idempotency
What is idempotency, you ask me?
Idempotence in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application.
— https://en.wikipedia.org/wiki/Idempotence
Your API can be called multiple times with the same of data — the user can rage tap the “Make Payment” button twice or thrice or some service might retry the same request from a DLQ even though it succeeded the first time. What do you do in these cases? Debit the user’s account twice? Hell, no.
That’s where you need to check for an Idempotency key to make sure that you don’t perform the same action twice for the same data.
Idempotency plays a key role in error handling and recovery mechanisms. It allows systems to retry operations without the risk of causing unintended state changes, which is especially important in distributed systems where network reliability can be an issue.
If you are working in an already-setup mature codebase, it’s most likely that the other developers have put in a idempotency checking mechanism in the place and you just need to use it.
Analytics Requirements
As developers, we have to bring visibility to the product managers, business stakeholders on what is going inside the system.
Metrics like what the API throughput was/is/will be do not matter to non-developers.
Product and Business folks want to get into visibility into how much orders were placed in a given time, how many times the app crashed, how much time it took user to sign up, etc.
Analytics play a very important role in helping decide the next decisions for the stakeholders.
As a backend developer, you should talk to your team lead or even better, to the product manager/owner about what kind of data and analytics they would like to see so that it can help in their future product decisions.
Anything you’d like me to add? Feel free to DM me on Twitter or email me shivamsinghal5432 [AT] gmail [DOT] com
I am always up for learning something new xD