9

My curiosity dragged me to find what is the stack, that Stack Overflow is based on, caching in particular. And the site here , gave description of how things are build up. But this gives the deployment knowledge of items , the hardware and software in use and how they are organized.But not how a particular feature is designed for example the badge allocation, We can see when you vote the the badge allocation takes a little time to reflect

Can somebody help me understand, how Redis is used?

  • Diagrams ( on vertical or horizontal scaling for distribution ), since the post mentions about 5 different cache levels.
  • Caching methodology ( write behind , write through etc), they also use in memory data structure along with Redis,
  • May be little bit on the data structure for the badge allocation for example.
  • Dataflow , for the data structures involved in in-memory and Redis with database

1 Answer 1

18

Diagram... ish:

[Web Server]                 [Web Server n]
(level 1, in process)        (level 1, in process)
      |                              |
      |       (on cache miss)        |
      |                              |
      +---------[Redis Master]-------+
                 (level 2)
                      |
                      |  (replication)
                      |
                [Redis Slave(s)]

We use asp.net cache to provide an in-memory level 1 cache. If that is empty, we check redis as level 2, before querying the actual data and storing it into both level 2. Either way, it is also stored at level 1. Redis supports data expiration natively, allowing things to fall out of the cache natively, but in the event of prompt updates being necessary, redis pub/sub is used to broadcast a "forget about this key" message to all web-servers.

That's... kinda about it. We don't over-complicate it, because it already works. For completeness, we also use additional slaves to ensure we have the data in a secondary data center, in case anyone should accidentally plug our cage into the same UPS twice, which as we all know would never happen, right?

3
  • If I update a key on WebServer1 and it sends a pub/sub message out to the other webservers (presumably including himself) to forget about a key, how does WebServer1 know that the message came from himself and to ignore it? Do you have some kind of source id that identifies the pub/sub sender?
    – Vince
    Apr 25, 2018 at 15:20
  • 1
    @Vince yes, we send a token that uniquely identifies the origin; however, you might also be interested to hear that Salvatore announced earlier today (redisconf18) that this kind of multi-layer cache scenario with protocol enhancements to directly to support it is one of the things they're looking at for the next list of things. If that happens, we'll almost certainly update SE.Redis to leverage that functionality Apr 25, 2018 at 19:57
  • Thank you! SE.Redis has been my client of choice.
    – Vince
    Apr 25, 2018 at 20:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .