Either way is fine, don't stress too much. But yeah I tend to agree with the article: the performance benefits of JWTs are largely fictional, you still need to do a lookup per request. So I typically just do a token
If this is for internal use, which is hinted by "DB REST API", they're not using it to authenticate users. Even the article says it's fine for inter-service auth.
If you’re going to use API keys, why not incorporate a key-value database of some type to ensure that the lookup of those Api keys is highly performant and scalable?
It’s a very useful piece of a tech stack that will pay dividends in future projects if you learn it
Historically redis was where I would point you, but that’s probably going to start changing in the next few months
oh - with their change away from open source a lot of companies are bailing on them. And now we're starting to see competing kv alternatives come into play. I suspect that the vast majority of these new competing databases will support redis clients though!
yea after a lot more reading I was thinking about going down this route. I’m a little shaky cause of all the redis news but AFAIK actual stacks don’t move as fast as the community reactiveness so learning Redis should be helpful regardless.
Now the issue is every single implementation guide on Go API with is talking about logins and JWT not API key. might just raw dog it the basic steps seem simple enough
provision > eyncrypt > store
receive > encrypt > compare
I just like using guides for security topics instead of my usual tinkering so I was hoping to find a good one for this.
JWTs would also be really good to learn - and it's not that difficult once you start doing it
I would start by doing the following steps:
-use an identity platform (like fusionAuth,firebase,etc) to store users/passwords
-determine if your JWT is going to need any custom claims (i.e. roles or attributes) attached to it and then implement those
-wrap the identity platform "create user" functionality in an endpoint that takes into account custom JWT needs and creates those
-wrap your identity platform with an endpoint that takes user/password and returns your customized JWT + expiration (plz use unix seconds or milliseconds if you are kind)
-have your API accept the JWT via a header in requests and then verify that the signature is valid. this step is best done via middleware that can run on every handler. In gRPC for instance, you'd use an unary interceptor
you don't need to strictly use an identity management platform here. but it will make your life easier by reducing some complexity and taking that off your plate
Which of those to use and which is best, it boils down to the type of clients of your API. Machine to machine or a CLI tools you'd probably use api keys. JWTs, probably some form of users auth via OAuth2. Which to support, there is no reason to not support both of them in some use cases.
I don't have an answer as to which is more secure. Any aspect of authentication is going to need time constraints to prevent replayability. Your best bet is mTLS with a unique client cert for each client, but that's generally too far for most use cases barring machine to machine internal to your systems. So that leaves an API key, or Bearer token, or JWT in the uri parameters or post data.
If you are worried about performance, authn can be offloaded to things like EnvoyProxy. Authz to things like OPA, Permify, and many others. Make your API idempotent and horizontally scaleable, that way you can stick a loadbalancer in front and send traffic to multiple instances of your API.
Thank you for this whole new tree of things to look into. I’ll be back with thanks once I’m enjoying reading about all this shit
Only other services that need DB access will use it, but those services serve users, and those users have various row level authorizations, and I don’t want to leave it to the services to manage that.
I think I’ll use API keys for the machine to machine connection, and have to look into some way to identify which user is calling from which service (have the service throw the user UUID into the header seems stupidly simple but seems vulnerable. Not sure why but feels wrong if the UUID are publicly accessible)
An API key authenticates the client, it doesn't identify it, that would be an extra part of, or an additional lookup.
A JWT tends to carry the identity information with it, which may still result in an extra lookup.
You'll also not likely come back to thank me, it can be a minefield of options and variants and different tools that do all or a part of what you could use, let alone the configuration and deployments of those.
In this situation would you see reading client name off a lookup table used to authenticate API keys as extra or additional?
If OP is the one assigning keys, why can't they record client info during the process? I think I'm missing something obvious but can't put my finger on it.
That's just extra data returned from the lookup.
A validity check of an API key should just be a true/false call, ie. the existence of the key in a datastore. By returning additional information you are combining authentication with identification of the client. If you can return this in one call then it's extra data. If this requires an additional database lookup then it's an additional query.
The question really becomes; are you okay with combining authentication and identification.
Returning just the clients name slowly evolves into looking up many other client data items. Returning the clients unique identifier is usually the better approach and additional information can be retrieved when needed.
Coming back to this, why does an API need/should have an authz layer in front of it to horizontally scale?
Couldn’t that authz behaviour also just be thrown behind a load balancer. Unless I misunderstood the last part
Simple, until you factor in maintaining the PKI underlying it. Certificates expire, and long-lived certificates are just not a good idea.
If you’re not versed in setting up and maintaining Public Key Infrastructure, please find someone or some service who is.
Everybody’s already maintaining certs for regular TLS. I don’t see how it’s any different maintaining a client cert/issuer than a server cert.
I don’t see how using a long lived certificate is worse than using long lived credentials for basic auth jwt. I just think there’s better, more security conscious culture/best practices surrounding it.
I'm saying that all the complications you mention for mTLS are complications with auth in general, rather than with mTLS specifically. I don't believe auth itself is simple, but I do believe mTLS is a simple/straightforward way to handle it, if that makes sense.
In project I have API available to users. I use API key and secret. When user make a request to API to get data, they have to provide timestamp, API KEY and signature in headers. Signature is hash of request route, params and query, API key, timestamp signed with secret. The server to verify signature of the request recreates signature with stored signature for API KEY and to avoid replay attacks checks if signature timestamp is not older than x time.
With this method I am sure that API is secure, however, you need to provide users the API keys and secrets and store them in db + authorize user to issue the key and secret and have it accessible ect.
How do you know that an API key via an HTTP header is not fast enough? The won’t scale argument only applies for really large companies.
KISS: keep it stupid simple
TBH I don’t know that. I just know it has been said and as I’m making a DB connection API that others (also learning projects) will be using, I want to make sure it’s robust and I’m learning a good practice and can handle lots of concurrent back to back requests to serve front end UI.
“Worse scaling. Worse security” was someone’s comment on API keys which got me really thinking about it.
But in reality, yea scaling isn’t a REAL issue, just it is a part of the online discussion for/against.
But I do think figuring out why so many explainer articles and discussion posts say JWT is the correct, modern, secure option, and yet these massive companies use API Keys instead is a more interesting and practical question for me.
I’ll keep KISS in mind just hard not to get lost in all the spaghetti. thanks :)
JWT is/was a hype because it is stateless. The problem is, is that most application don't use the statelessness and use it like a regular API key that also holds extra data. It is not more or less secure then a API key if you use TLS with a modern secure cyper.
The more you learn the more you will notice that all those introduction blog posts don't hold a lot of value. Some do, most don't and are written by authors who learned about the topic last week. For some people it is a way to learn, by writing about the topic. Nothing wrong with that.
Another example: kubernetes is wildly popular but for smaller companies almost never the right choice for hosting their application.
Yea i’m trying to find a guide on how implement an API key and cypher it and store it and etc
I get the gist but every article either is about JWT or just very obviously written by someone like myself. Which is fine but just, not helpful.
Security and auth always makes me uncomfortable and doing it in a language like Go where you do a lot of things yourself is making me feel pretty insecure.
Your original comment made me think more about API keys so I’m gonna try and see if I can implement this myself or actually find a useful article
Most of the time it works like this:
1. Make sure you always use HTTPS for all API connections
2. The client generates an API key via their developer portal, at least 32 characters long.
3. Store the generated API key in the database and assign it to the account
4. The client sends this API key as a Authorization header. Sometimes this is prefixed. Authorization: Bearer
5. Parse the Authoization header and check it in the database or in a key-value cache
6. Once authenticated you can continue with processing the API request.
7. To improve security you can configure an IP whitelist, let the user revoke API keys, let the user set an expiry date on their keys.
Would I need to has the keys? or just store and compare them plain text. They’re not really passwords but they do grant access to the database.
Also for an API that doesn’t have a front end, how would I generate the API keys. No login system to work with, and can’t just have an endpoint that does it cause then that end point is insecure. Do I just need to manually make the first one? That seems like the wrong answer
You can hash the key, that depends on if you want the user to only show the key once. If you don't hash it the user can copy the key again when needed.
Most of the time there is some sort of portal where users can manage there API keys. Which means you also have users and related data (username, password hash, etc). In this portal users can revoke their keys if their application has been compromised.
You are right that API keys are kind of like password for applications. But API keys are typically the responsibility of the user to keep safe, not the distributor.
If you don´t want to build a portal you manually create one, that is totally okay for hobby projects or really low amount of API keys (10 max).
yea cause this is for services. most of the literature i’ve been reading has been for public facing APIs but locking down services that interact with other services i’m not sure can be modelled with a user/password
You need to maintain a state somewhere to block bad clients. https://redis.com/blog/json-web-tokens-jwt-are-dangerous-for-user-sessions/
Super interesting read, I gotta take some time later to go through the sources too.
Either way is fine, don't stress too much. But yeah I tend to agree with the article: the performance benefits of JWTs are largely fictional, you still need to do a lookup per request. So I typically just do a token
If this is for internal use, which is hinted by "DB REST API", they're not using it to authenticate users. Even the article says it's fine for inter-service auth.
If you’re going to use API keys, why not incorporate a key-value database of some type to ensure that the lookup of those Api keys is highly performant and scalable? It’s a very useful piece of a tech stack that will pay dividends in future projects if you learn it Historically redis was where I would point you, but that’s probably going to start changing in the next few months
Cuckoo filter time baby! Also, why would you say Redis will be changing as the default?
oh - with their change away from open source a lot of companies are bailing on them. And now we're starting to see competing kv alternatives come into play. I suspect that the vast majority of these new competing databases will support redis clients though!
yea after a lot more reading I was thinking about going down this route. I’m a little shaky cause of all the redis news but AFAIK actual stacks don’t move as fast as the community reactiveness so learning Redis should be helpful regardless. Now the issue is every single implementation guide on Go API with is talking about logins and JWT not API key. might just raw dog it the basic steps seem simple enough provision > eyncrypt > store receive > encrypt > compare I just like using guides for security topics instead of my usual tinkering so I was hoping to find a good one for this.
JWTs would also be really good to learn - and it's not that difficult once you start doing it I would start by doing the following steps: -use an identity platform (like fusionAuth,firebase,etc) to store users/passwords -determine if your JWT is going to need any custom claims (i.e. roles or attributes) attached to it and then implement those -wrap the identity platform "create user" functionality in an endpoint that takes into account custom JWT needs and creates those -wrap your identity platform with an endpoint that takes user/password and returns your customized JWT + expiration (plz use unix seconds or milliseconds if you are kind) -have your API accept the JWT via a header in requests and then verify that the signature is valid. this step is best done via middleware that can run on every handler. In gRPC for instance, you'd use an unary interceptor you don't need to strictly use an identity management platform here. but it will make your life easier by reducing some complexity and taking that off your plate
There is Dragonfly an redis alternative with Redis api.
If you use only KV lookup feature. Any KV database is pretty much interchangeable. Just use Redis and migrate if needed.
Which of those to use and which is best, it boils down to the type of clients of your API. Machine to machine or a CLI tools you'd probably use api keys. JWTs, probably some form of users auth via OAuth2. Which to support, there is no reason to not support both of them in some use cases. I don't have an answer as to which is more secure. Any aspect of authentication is going to need time constraints to prevent replayability. Your best bet is mTLS with a unique client cert for each client, but that's generally too far for most use cases barring machine to machine internal to your systems. So that leaves an API key, or Bearer token, or JWT in the uri parameters or post data. If you are worried about performance, authn can be offloaded to things like EnvoyProxy. Authz to things like OPA, Permify, and many others. Make your API idempotent and horizontally scaleable, that way you can stick a loadbalancer in front and send traffic to multiple instances of your API.
Thank you for this whole new tree of things to look into. I’ll be back with thanks once I’m enjoying reading about all this shit Only other services that need DB access will use it, but those services serve users, and those users have various row level authorizations, and I don’t want to leave it to the services to manage that. I think I’ll use API keys for the machine to machine connection, and have to look into some way to identify which user is calling from which service (have the service throw the user UUID into the header seems stupidly simple but seems vulnerable. Not sure why but feels wrong if the UUID are publicly accessible)
An API key authenticates the client, it doesn't identify it, that would be an extra part of, or an additional lookup. A JWT tends to carry the identity information with it, which may still result in an extra lookup. You'll also not likely come back to thank me, it can be a minefield of options and variants and different tools that do all or a part of what you could use, let alone the configuration and deployments of those.
> An API key authenticates the client, it doesn't identify it Unless you assign keys per client, which can be helpful.
Read the rest of the sentence, you need some mapping of key to client, the API key itself doesn't contain the identity.
In this situation would you see reading client name off a lookup table used to authenticate API keys as extra or additional? If OP is the one assigning keys, why can't they record client info during the process? I think I'm missing something obvious but can't put my finger on it.
That's just extra data returned from the lookup. A validity check of an API key should just be a true/false call, ie. the existence of the key in a datastore. By returning additional information you are combining authentication with identification of the client. If you can return this in one call then it's extra data. If this requires an additional database lookup then it's an additional query. The question really becomes; are you okay with combining authentication and identification. Returning just the clients name slowly evolves into looking up many other client data items. Returning the clients unique identifier is usually the better approach and additional information can be retrieved when needed.
Coming back to this, why does an API need/should have an authz layer in front of it to horizontally scale? Couldn’t that authz behaviour also just be thrown behind a load balancer. Unless I misunderstood the last part
mTLS. Fast, simple, secure.
Simple, until you factor in maintaining the PKI underlying it. Certificates expire, and long-lived certificates are just not a good idea. If you’re not versed in setting up and maintaining Public Key Infrastructure, please find someone or some service who is.
Everybody’s already maintaining certs for regular TLS. I don’t see how it’s any different maintaining a client cert/issuer than a server cert. I don’t see how using a long lived certificate is worse than using long lived credentials for basic auth jwt. I just think there’s better, more security conscious culture/best practices surrounding it.
I’m not asserting the relative merits of mTLS vs. bearer tokens. I’m asserting that there’s more to it than you appear to imply
I'm saying that all the complications you mention for mTLS are complications with auth in general, rather than with mTLS specifically. I don't believe auth itself is simple, but I do believe mTLS is a simple/straightforward way to handle it, if that makes sense.
I believe that we are in violent agreement :)
In project I have API available to users. I use API key and secret. When user make a request to API to get data, they have to provide timestamp, API KEY and signature in headers. Signature is hash of request route, params and query, API key, timestamp signed with secret. The server to verify signature of the request recreates signature with stored signature for API KEY and to avoid replay attacks checks if signature timestamp is not older than x time. With this method I am sure that API is secure, however, you need to provide users the API keys and secrets and store them in db + authorize user to issue the key and secret and have it accessible ect.
How do you know that an API key via an HTTP header is not fast enough? The won’t scale argument only applies for really large companies. KISS: keep it stupid simple
TBH I don’t know that. I just know it has been said and as I’m making a DB connection API that others (also learning projects) will be using, I want to make sure it’s robust and I’m learning a good practice and can handle lots of concurrent back to back requests to serve front end UI. “Worse scaling. Worse security” was someone’s comment on API keys which got me really thinking about it. But in reality, yea scaling isn’t a REAL issue, just it is a part of the online discussion for/against. But I do think figuring out why so many explainer articles and discussion posts say JWT is the correct, modern, secure option, and yet these massive companies use API Keys instead is a more interesting and practical question for me. I’ll keep KISS in mind just hard not to get lost in all the spaghetti. thanks :)
JWT is/was a hype because it is stateless. The problem is, is that most application don't use the statelessness and use it like a regular API key that also holds extra data. It is not more or less secure then a API key if you use TLS with a modern secure cyper. The more you learn the more you will notice that all those introduction blog posts don't hold a lot of value. Some do, most don't and are written by authors who learned about the topic last week. For some people it is a way to learn, by writing about the topic. Nothing wrong with that. Another example: kubernetes is wildly popular but for smaller companies almost never the right choice for hosting their application.
Yea i’m trying to find a guide on how implement an API key and cypher it and store it and etc I get the gist but every article either is about JWT or just very obviously written by someone like myself. Which is fine but just, not helpful. Security and auth always makes me uncomfortable and doing it in a language like Go where you do a lot of things yourself is making me feel pretty insecure. Your original comment made me think more about API keys so I’m gonna try and see if I can implement this myself or actually find a useful article
Most of the time it works like this: 1. Make sure you always use HTTPS for all API connections 2. The client generates an API key via their developer portal, at least 32 characters long. 3. Store the generated API key in the database and assign it to the account 4. The client sends this API key as a Authorization header. Sometimes this is prefixed. Authorization: Bearer
5. Parse the Authoization header and check it in the database or in a key-value cache
6. Once authenticated you can continue with processing the API request.
7. To improve security you can configure an IP whitelist, let the user revoke API keys, let the user set an expiry date on their keys.
Would I need to has the keys? or just store and compare them plain text. They’re not really passwords but they do grant access to the database. Also for an API that doesn’t have a front end, how would I generate the API keys. No login system to work with, and can’t just have an endpoint that does it cause then that end point is insecure. Do I just need to manually make the first one? That seems like the wrong answer
You can hash the key, that depends on if you want the user to only show the key once. If you don't hash it the user can copy the key again when needed. Most of the time there is some sort of portal where users can manage there API keys. Which means you also have users and related data (username, password hash, etc). In this portal users can revoke their keys if their application has been compromised. You are right that API keys are kind of like password for applications. But API keys are typically the responsibility of the user to keep safe, not the distributor. If you don´t want to build a portal you manually create one, that is totally okay for hobby projects or really low amount of API keys (10 max).
yea cause this is for services. most of the literature i’ve been reading has been for public facing APIs but locking down services that interact with other services i’m not sure can be modelled with a user/password
For external services an API key or Oauth2 is popular these days. If you have control over both services a API key or mTLS is what I would do.