Redis Multi-Master Replication Announcing the arrival of Valued Associate #679: Cesar Manara ...
Plotting a Maclaurin series
NIntegrate on a solution of a matrix ODE
Is it OK to use the testing sample to compare algorithms?
Is there a verb for listening stealthily?
Noise in Eigenvalues plot
By what mechanism was the 2017 UK General Election called?
Short story about astronauts fertilizing soil with their own bodies
How do you write "wild blueberries flavored"?
What was the last profitable war?
Diophantine equation 3^a+1=3^b+5^c
Why do C and C++ allow the expression (int) + 4*5;
Random body shuffle every night—can we still function?
.bashrc alias for a command with fixed second parameter
Why does BitLocker not use RSA?
Why not use the yoke to control yaw, as well as pitch and roll?
Why complex landing gears are used instead of simple, reliable and light weight muscle wire or shape memory alloys?
Where did Ptolemy compare the Earth to the distance of fixed stars?
How many time has Arya actually used Needle?
Did John Wesley plagiarize Matthew Henry...?
French equivalents of おしゃれは足元から (Every good outfit starts with the shoes)
Is the Mordenkainen's Sword spell underpowered?
How could a hydrazine and N2O4 cloud (or it's reactants) show up in weather radar?
New Order #6: Easter Egg
Is this Kuo-toa homebrew race balanced?
Redis Multi-Master Replication
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern)
Come Celebrate our 10 Year Anniversary!MySQL 5.1 Replication Topologies: Multi-MasterMySQL master-master-replication as master for master-slave-replicationmulti-master file replicationUSABLE multi-master replication for Postgres?OpenLDAP: recommended way to add a new server to a replicated multi-master setupStatic master in Redis replicationRedis - Master - Master Sync without shardingHow do i measure redis replication delay?Redis Master/Slave Replication - Permission denied connecting to masterWhy Redis slaves don't take over the master after master fail in Redis Cluster?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
Scenario: We have two datacentres which run concurrently (both serve traffic).
Each has it's own entire stack of infrastructure, so it can operate without the other being up.
That said, we would like, when network conditions allow for it, for our Redis database to be synchronised between the two.
Losing keys during a network partition is acceptable, as is having duplicate keys - it's only cache data.
But, we get the most benefit from the cache when both datacentres are up (from 15-20% cache hits, to 30-40% cache hits).
After some searching around I couldn't find anything that would effectively give us multi-master. (Mostly saying "don't do that" or "it's not supported").
In the end, I wrote a client that connects to both masters, subscribes to keyspace events for the database, and then pushes all the SET commands between DBs (all our keys are set with expiry) - with some internal "recently seen" cache to prevent a replay loop.
At the moment, that's working great - the only real downside is that after we get a new key event, we then have to issue a GETEX to get the key and it's expiry - so you end up with a second read (thus about 2x the RTT latency for the remote DC). It's also limited to really just supporting SET.
My question is: Is there a better way of getting this kind of multi-master replication?
I started looking at SYNC/PSYNC, but there's not a lot of documentation on these protocols, and not sure what obligations a client might have to not break the server.
replication redis
bumped to the homepage by Community♦ 10 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
Scenario: We have two datacentres which run concurrently (both serve traffic).
Each has it's own entire stack of infrastructure, so it can operate without the other being up.
That said, we would like, when network conditions allow for it, for our Redis database to be synchronised between the two.
Losing keys during a network partition is acceptable, as is having duplicate keys - it's only cache data.
But, we get the most benefit from the cache when both datacentres are up (from 15-20% cache hits, to 30-40% cache hits).
After some searching around I couldn't find anything that would effectively give us multi-master. (Mostly saying "don't do that" or "it's not supported").
In the end, I wrote a client that connects to both masters, subscribes to keyspace events for the database, and then pushes all the SET commands between DBs (all our keys are set with expiry) - with some internal "recently seen" cache to prevent a replay loop.
At the moment, that's working great - the only real downside is that after we get a new key event, we then have to issue a GETEX to get the key and it's expiry - so you end up with a second read (thus about 2x the RTT latency for the remote DC). It's also limited to really just supporting SET.
My question is: Is there a better way of getting this kind of multi-master replication?
I started looking at SYNC/PSYNC, but there's not a lot of documentation on these protocols, and not sure what obligations a client might have to not break the server.
replication redis
bumped to the homepage by Community♦ 10 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
Scenario: We have two datacentres which run concurrently (both serve traffic).
Each has it's own entire stack of infrastructure, so it can operate without the other being up.
That said, we would like, when network conditions allow for it, for our Redis database to be synchronised between the two.
Losing keys during a network partition is acceptable, as is having duplicate keys - it's only cache data.
But, we get the most benefit from the cache when both datacentres are up (from 15-20% cache hits, to 30-40% cache hits).
After some searching around I couldn't find anything that would effectively give us multi-master. (Mostly saying "don't do that" or "it's not supported").
In the end, I wrote a client that connects to both masters, subscribes to keyspace events for the database, and then pushes all the SET commands between DBs (all our keys are set with expiry) - with some internal "recently seen" cache to prevent a replay loop.
At the moment, that's working great - the only real downside is that after we get a new key event, we then have to issue a GETEX to get the key and it's expiry - so you end up with a second read (thus about 2x the RTT latency for the remote DC). It's also limited to really just supporting SET.
My question is: Is there a better way of getting this kind of multi-master replication?
I started looking at SYNC/PSYNC, but there's not a lot of documentation on these protocols, and not sure what obligations a client might have to not break the server.
replication redis
Scenario: We have two datacentres which run concurrently (both serve traffic).
Each has it's own entire stack of infrastructure, so it can operate without the other being up.
That said, we would like, when network conditions allow for it, for our Redis database to be synchronised between the two.
Losing keys during a network partition is acceptable, as is having duplicate keys - it's only cache data.
But, we get the most benefit from the cache when both datacentres are up (from 15-20% cache hits, to 30-40% cache hits).
After some searching around I couldn't find anything that would effectively give us multi-master. (Mostly saying "don't do that" or "it's not supported").
In the end, I wrote a client that connects to both masters, subscribes to keyspace events for the database, and then pushes all the SET commands between DBs (all our keys are set with expiry) - with some internal "recently seen" cache to prevent a replay loop.
At the moment, that's working great - the only real downside is that after we get a new key event, we then have to issue a GETEX to get the key and it's expiry - so you end up with a second read (thus about 2x the RTT latency for the remote DC). It's also limited to really just supporting SET.
My question is: Is there a better way of getting this kind of multi-master replication?
I started looking at SYNC/PSYNC, but there's not a lot of documentation on these protocols, and not sure what obligations a client might have to not break the server.
replication redis
replication redis
asked Aug 28 '15 at 5:30
user65539
bumped to the homepage by Community♦ 10 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 10 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
I am also looking for a solution to this challenge, so far, I found:
https://github.com/Netflix/dynomite/wiki/Architecture
And maybe...
https://github.com/CodisLabs/codis or https://github.com/twitter/twemproxy
answered Aug 11 '17 at 13:16
nadavkavnadavkav
1,06985
Unfortunately none of these support the full level of commands, and for the most part don't do the 'best effort' idea, they're more clustering options than anything else.
– user65539
Aug 15 '17 at 2:49
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "2"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f717406%2fredis-multi-master-replication%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I am also looking for a solution to this challenge, so far, I found:
https://github.com/Netflix/dynomite/wiki/Architecture
And maybe...
https://github.com/CodisLabs/codis or https://github.com/twitter/twemproxy
answered Aug 11 '17 at 13:16
nadavkavnadavkav
1,06985
Unfortunately none of these support the full level of commands, and for the most part don't do the 'best effort' idea, they're more clustering options than anything else.
– user65539
Aug 15 '17 at 2:49
add a comment |
I am also looking for a solution to this challenge, so far, I found:
https://github.com/Netflix/dynomite/wiki/Architecture
And maybe...
https://github.com/CodisLabs/codis or https://github.com/twitter/twemproxy
answered Aug 11 '17 at 13:16
nadavkavnadavkav
1,06985
Unfortunately none of these support the full level of commands, and for the most part don't do the 'best effort' idea, they're more clustering options than anything else.
– user65539
Aug 15 '17 at 2:49
add a comment |
I am also looking for a solution to this challenge, so far, I found:
https://github.com/Netflix/dynomite/wiki/Architecture
And maybe...
https://github.com/CodisLabs/codis or https://github.com/twitter/twemproxy
answered Aug 11 '17 at 13:16
nadavkavnadavkav
1,06985
I am also looking for a solution to this challenge, so far, I found:
https://github.com/Netflix/dynomite/wiki/Architecture
And maybe...
https://github.com/CodisLabs/codis or https://github.com/twitter/twemproxy
answered Aug 11 '17 at 13:16
nadavkavnadavkav
1,06985
answered Aug 11 '17 at 13:16
nadavkavnadavkav
1,06985
answered Aug 11 '17 at 13:16
nadavkavnadavkav
1,06985
answered Aug 11 '17 at 13:16
nadavkavnadavkav
1,06985
1,06985
Unfortunately none of these support the full level of commands, and for the most part don't do the 'best effort' idea, they're more clustering options than anything else.
– user65539
Aug 15 '17 at 2:49
add a comment |
Unfortunately none of these support the full level of commands, and for the most part don't do the 'best effort' idea, they're more clustering options than anything else.
– user65539
Aug 15 '17 at 2:49
Unfortunately none of these support the full level of commands, and for the most part don't do the 'best effort' idea, they're more clustering options than anything else.
– user65539
Aug 15 '17 at 2:49
Unfortunately none of these support the full level of commands, and for the most part don't do the 'best effort' idea, they're more clustering options than anything else.
– user65539
Aug 15 '17 at 2:49
add a comment |
Thanks for contributing an answer to Server Fault!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f717406%2fredis-multi-master-replication%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
