[PLUG] help with /proc/cpuinfo

VY vyau5678 at gmail.com
Mon Apr 29 16:37:25 UTC 2019


Thanks Ben, Richard.
All the points both of you raised are very valid.   Unfortunately, I am
just an engineer on the development team and managing resources are
something I do not own and have no control.  However, while monitoring
runtime applications that we developed, I noticed that disparity in load
for that one systems (out of a total of 4).   They are all running
identical applications and I ruled out most other things that I can measure
without root/sudo.
So when I was told there are a busy VM running on a same box that spills
its problems to other neighbors, I decided to send an email to get a
confirmation
(and also getting a better explanation) here.

Honestly, I found this resource allocation scheme very unfair.   We do not
run very intensive apps and this image is exactly size/shape as the others
but yet it has several order of magnitude in terms of load (while handling
similar amount of traffic).   This image is being penalized for something
another image is doing.    Not sure how difficult to reconfigure but I was
told there's not much the system team can do.   I do not have VM
reconfiguration experience
so not sure how to tackle this.

While we are on the subject, is it hard to get a local box, deploy Xen and
deploy a few image?   If anything, I am curious how these things work and
best of all how to allocate resources fairly?   I have cheap laptops at
home that I can spare but not sure if they are powerful enough.   This box
at work is a Xeon so my home machines will not be a match for certain.


-v



On Sun, Apr 28, 2019 at 4:17 PM Richard England <rlengland at frontier.com>
wrote:

>
> On 4/28/19 3:25 PM, Ben Koenig wrote:
> > Forgive my ignorance in resurrecting this, but isn't the primary
> > purpose of a VM to partition system resources?
> >
> > Is this "noisy neighbor" problem a side effect of using VM's, or just
> > bad VM management on the part of the host? This whole Cloud idea seems
> > pretty pointless if a single VM is able to consume CPU time in the
> > same way a normal process does. You might as well just give everyone a
> > user account and let them fight over a shared pool of RAM.....
> >
> >
> > Just curious to know if the path forward in a situation like this is
> > to blame the configuration or the technology itself. Would this be any
> > different on a different hypervisor...etc.
> >
> >
> > On 4/25/19 1:17 PM, VY wrote:
> >> Hi Aaron
> >>
> >> Thanks for confirming.?? I do not yet know how to troubleshoot in a
> >> Xen env
> >> but now that you
> >> dissected the data with me (which no one on our team has so far), I
> >> understand the situation now.
> >>
> >> There's no much I can do, and we do not have another place to migrate
> >> this
> >> image.
> >> Oh well....
> >>
> >> thanks again!
> >>
> >> -v
> >>
> >>
> >> On Thu, Apr 25, 2019 at 1:05 PM Aaron Burt <aaron at bavariati.org> wrote:
> >>
> >>> On 2019-04-25 10:54, VY wrote:
> >>>> Yes, I love to learn as well.
> >>>>
> >>>> This is the output to lscpu:
> >>>> ???? Architecture:?????????????????? x86_64
> >>> [...]
> >>>> Hypervisor vendor:???????? Xen
> >>>> Virtualization type:???? full
> >>> Ah-hah.?? You're in a VM, and I'll bet you have a "noisy neighbor."
> >>>
> >>>> The load average is:
> >>>> load average: 464.68, 415.14, 416.96
> >>>> which does not make sense at all.
> >>> Loadavg is just how many processes are waiting to use the CPU.
> >>>
> >>>> The rest of TOP:
> >>>> ???? Cpu(s): 51.3%us, 16.0%sy,?? 0.0%ni, 32.0%id,?? 0.0%wa, 0.0%hi,
> >>>> 0.4%si,
> >>>> 0.2%st
> >>>>
> >>>> If I hit 1, it affects all 4 CPUs.
> >>> All good.?? Very much looks like a "noisy neighbor" problem, which is
> >>> when another VM on the hypervisor is hogging all the CPU (or RAM) and
> >>> leaving you with no compute resources.?? From your VM's perspective,
> >>> it's
> >>> going at the rated clock speed, but time is going by REALLY FAST.
> >>>
> >>>> Can you elaborate on why
> >>>> ?? >?????? apicid : 25
> >>>>> ???? initial apicid : 25
> >>>> 25 is a weird number????? From an earlier thread, is this simply a
> >>>> logical
> >>>> ID?
> >>> Eh, sort of.?? There should only be a couple APICs in the system.?? And
> >>> usually it'll be pretty consistent.
> >>> ???? But since it's a VM all bets are off.
> >>>
> >>>> All the other systems are reporting this number as 4 and all of them
> >>>> are
> >>>> having reasonable load.
> >>> They're on a different hypervisor machine, and probably a different
> >>> version of the hypervisor software.
> >>>
> >>>> I do not have root access nor sudo.?? I want to try and find out why
> >>>> the load is so high before I escalate and argue for more privilege.
> >>>> When I brought this up to the responsible team, I was given a probable
> >>>> cause -- There are other activities hosting this VM server and they
> >>>> are
> >>>> causing this issue.
> >>> So they already told you that you have a noisy neighbor.?? All right
> >>> then.?? Don't use that VM and get by on the 3 you have, or ask the team
> >>> to migrate your slow VM to a different hypervisor machine, or ask for a
> >>> new VM on a less loaded hypervisor machine.?? But unless the noisy
> >>> neighbor calms down it sounds like your one sad VM isn't getting any
> >>> better.
> >>>
> >>> Is this a customer-facing service??? If so, you should point this out
> to
> >>> your hosting team.
> >>>
> >>> Good luck,
> >>> ?????? Aaron
> >>> _______________________________________________
> >>> PLUG mailing list
> >>> PLUG at pdxlinux.org
> >>> http://lists.pdxlinux.org/mailman/listinfo/plug
> >>>
> >> _______________________________________________
> >> PLUG mailing list
> >> PLUG at pdxlinux.org
> >> http://lists.pdxlinux.org/mailman/listinfo/plug
> > _______________________________________________
> > PLUG mailing list
> > PLUG at pdxlinux.org
> > http://lists.pdxlinux.org/mailman/listinfo/plug
>
> There are only so many "clock ticks" to go around on any system. The
> purpose of a VM is to distribute those as efficiently as possible.
> However, if one of the "neighbors" is able to consume too much time, or??
> there are a lot of them, then a reconfiguration for load balancing is in
> order.?? As was already pointed out, this is something the managers of
> your VM/Hypervisor needs to address. if they are unable or unwilling,
> there isn't much you can do except have them move you to an alternative??
> server or find a different hosting service that will respond to your needs.
>
> The bottom line: is the under-performance of the one server a detriment
> to the site(s) or service(s) you are providing or is it just an
> annoyance??? You will probably have to document the impact of the one
> server on your users and see if your provider will address it.
>
> ~~R
>
>
> _______________________________________________
> PLUG mailing list
> PLUG at pdxlinux.org
> http://lists.pdxlinux.org/mailman/listinfo/plug
>



More information about the PLUG mailing list