GKE periodic feezes with no logsChange KUBE_MAX_PD_VOLS env var in master GKEHow to enable Stackdriver...
Are there other characters in the Star Wars universe who had damaged bodies and needed to wear an outfit like Darth Vader?
Would the melodic leap of the opening phrase of Mozart's K545 be considered dissonant?
Draw bounding region by list of points
How to mitigate "bandwagon attacking" from players?
Misplaced tyre lever - alternatives?
Why would the IRS ask for birth certificates or even audit a small tax return?
Quitting employee has privileged access to critical information
GPL code private and stolen
How to merge row in the first column in LaTeX
Where is this quote about overcoming the impossible said in "Interstellar"?
Wardrobe above a wall with fuse boxes
If there are any 3nion, 5nion, 7nion, 9nion, 10nion, etc.
Where is the fallacy here?
Book about a time-travel war fought by computers
How do you say “my friend is throwing a party, do you wanna come?” in german
Sometimes a banana is just a banana
In which way proportional valves are controlled solely by current?
What is the meaning of "notice to quit at once" and "Lotty points”
If nine coins are tossed, what is the probability that the number of heads is even?
PTIJ: What’s wrong with eating meat and couscous?
Must 40/100G uplink ports on a 10G switch be connected to another switch?
Is divide-by-zero a security vulnerability?
Why doesn't "adolescent" take any articles in "listen to adolescent agonising"?
Is there a full canon version of Tyrion's jackass/honeycomb joke?
GKE periodic feezes with no logs
Change KUBE_MAX_PD_VOLS env var in master GKEHow to enable Stackdriver Monitoring and Stackdriver trace on existing GKE cluster?Detecting Kubernetes OOMKilled Events in GKE LogsBroken GKE Backend Health Check DefaultGKE pod can't make any DNS requestsGKE Internal Server errorKubernetes (on GKE) external connection through NAT for specific kube services?Changing GKE cluster subnetManaging group memberships in kubernetes (GKE)Enable VPC-native (alias IP) on an existing GKE cluster
I have several GKE clusters currently running in us-east1. They are all regionally deployed with nodes in each zone. Since last Thursday ( 2/28/19 ) I have been experiencing intermittent freezes on the cluster where the the cluster does not appear to be getting traffic. The pods on the cluster appear to restart as the downtime is usually just a minute, but there is no log or record of the downtime. I have noticed that datadog will stop reporting metrics during the issue leading me to believe that this may be network related. This seems to happen on all of my clusters, but to varying degrees of downtime. Some of my processes appear to continue to work, but are slow, while other just stop responding altogether.
I am a bit at a loss as to what would be causing multiple clusters to simultaneously have issues spawning new pods and stop sending data. Any thoughts on where I can look for logs or records to help figure out the issue?
New contributor
Daniel Picard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
I have several GKE clusters currently running in us-east1. They are all regionally deployed with nodes in each zone. Since last Thursday ( 2/28/19 ) I have been experiencing intermittent freezes on the cluster where the the cluster does not appear to be getting traffic. The pods on the cluster appear to restart as the downtime is usually just a minute, but there is no log or record of the downtime. I have noticed that datadog will stop reporting metrics during the issue leading me to believe that this may be network related. This seems to happen on all of my clusters, but to varying degrees of downtime. Some of my processes appear to continue to work, but are slow, while other just stop responding altogether.
I am a bit at a loss as to what would be causing multiple clusters to simultaneously have issues spawning new pods and stop sending data. Any thoughts on where I can look for logs or records to help figure out the issue?
New contributor
Daniel Picard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
I have several GKE clusters currently running in us-east1. They are all regionally deployed with nodes in each zone. Since last Thursday ( 2/28/19 ) I have been experiencing intermittent freezes on the cluster where the the cluster does not appear to be getting traffic. The pods on the cluster appear to restart as the downtime is usually just a minute, but there is no log or record of the downtime. I have noticed that datadog will stop reporting metrics during the issue leading me to believe that this may be network related. This seems to happen on all of my clusters, but to varying degrees of downtime. Some of my processes appear to continue to work, but are slow, while other just stop responding altogether.
I am a bit at a loss as to what would be causing multiple clusters to simultaneously have issues spawning new pods and stop sending data. Any thoughts on where I can look for logs or records to help figure out the issue?
New contributor
Daniel Picard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
I have several GKE clusters currently running in us-east1. They are all regionally deployed with nodes in each zone. Since last Thursday ( 2/28/19 ) I have been experiencing intermittent freezes on the cluster where the the cluster does not appear to be getting traffic. The pods on the cluster appear to restart as the downtime is usually just a minute, but there is no log or record of the downtime. I have noticed that datadog will stop reporting metrics during the issue leading me to believe that this may be network related. This seems to happen on all of my clusters, but to varying degrees of downtime. Some of my processes appear to continue to work, but are slow, while other just stop responding altogether.
I am a bit at a loss as to what would be causing multiple clusters to simultaneously have issues spawning new pods and stop sending data. Any thoughts on where I can look for logs or records to help figure out the issue?
New contributor
Daniel Picard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Daniel Picard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Daniel Picard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 6 mins ago
Daniel PicardDaniel Picard
1
1
New contributor
Daniel Picard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Daniel Picard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Daniel Picard is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "2"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Daniel Picard is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f957113%2fgke-periodic-feezes-with-no-logs%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Daniel Picard is a new contributor. Be nice, and check out our Code of Conduct.
Daniel Picard is a new contributor. Be nice, and check out our Code of Conduct.
Daniel Picard is a new contributor. Be nice, and check out our Code of Conduct.
Daniel Picard is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Server Fault!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f957113%2fgke-periodic-feezes-with-no-logs%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown