[OpenIndiana-discuss] KVM Failed to allocate memory: Resource temporarily unavailable
Fucai Liang （BLCT）
fcliang at baolict.com
Thu Dec 10 10:01:37 UTC 2015
Thanks for your help！
root at oi01:~# swap -sh
total: 8.1G allocated + 21M reserved = 8.1G used, 16G available
root at oi01:~# swap -l
swapfile dev swaplo blocks free
/dev/zvol/dsk/rpool/swap 100,2 8 12574712 12574712
root at oi01:~#
16G swap ， so KVM can locked 16G memory .
thanks again .
------------------------------
Fucai Liang
On Dec 10, 2015, at 17:24, Jonathan Adams <t12nslookup at gmail.com> wrote:
> how much swap do you have?
>
> KVM automatically allocates the same amount of swap as memory when you
> start an instance, if you don't have the free swap it won't work.
>
> Jon
>
> On 10 December 2015 at 04:04, Fucai Liang （BLCT） <fcliang at baolict.com>
> wrote:
>
>>
>>
>> Hello, guys：
>>
>> I has a server running oi_151.1.8, the server have 32G memory .
>>
>>
>> root at oi01:~# prtconf | grep Memory
>> Memory size: 32760 Megabytes
>> root at oi01:~# echo "::memstat" | mdb -k
>> Page Summary Pages MB %Tot
>> ------------ ---------------- ---------------- ----
>> Kernel 451736 1764 5%
>> ZFS File Data 78429 306 1%
>> Anon 19849 77 0%
>> Exec and libs 586 2 0%
>> Page cache 2170 8 0%
>> Free (cachelist) 10432 40 0%
>> Free (freelist) 7821076 30551 93%
>>
>> Total 8384278 32751
>> Physical 8384277 32751
>> root at oi01:~#
>>
>>
>> I lauch a KVM VM by
>>
>>
>> #!/bin/sh
>>
>> qemu-kvm \
>> -vnc 0.0.0.0:2 \
>> -cpu host \
>> -smp 2 \
>> -m 8192 \
>> -no-hpet \
>> -localtime \
>> -drive file=/dev/zvol/rdsk/rpool/svrinit,if=virtio,index=0 \
>> -net nic,vlan=0,name=e1000g0,model=e1000,macaddr=2:8:20:bd:ae:01 \
>> -net
>> vnic,vlan=0,name=e1000g0,ifname=vnic01,macaddr=2:8:20:bd:ae:01 \
>> -vga std \
>> -daemonize
>>
>>
>> root at oi01:~# echo "::memstat" | mdb -k
>> Page Summary Pages MB %Tot
>> ------------ ---------------- ---------------- ----
>> Kernel 461061 1801 5%
>> ZFS File Data 78446 306 1%
>> Anon 2131265 8325 25%
>> Exec and libs 918 3 0%
>> Page cache 2255 8 0%
>> Free (cachelist) 10015 39 0%
>> Free (freelist) 5700318 22266 68%
>>
>> Total 8384278 32751
>> Physical 8384277 32751
>> root at oi01:~#
>>
>>
>> then I launch second KVM VM,
>>
>> #!/bin/sh
>>
>> qemu-kvm \
>> -enable-kvm \
>> -vnc 0.0.0.0:3 \
>> -cpu host \
>> -smp 2 \
>> -m 8192 \
>> -no-hpet \
>> -localtime \
>> -drive file=/dev/zvol/rdsk/rpool/svr03,if=virtio,index=0 \
>> -net nic,vlan=0,name=e1000g0,model=e1000,macaddr=2:8:20:bd:ae:03 \
>> -net
>> vnic,vlan=0,name=e1000g0,ifname=vnic03,macaddr=2:8:20:bd:ae:03 \
>> -vga std \
>> -daemonize
>>
>>
>>
>> I got err message :
>> Failed to allocate memory: Resource temporarily unavailable
>>
>> -bash: fork: Not enough space
>>
>> .
>>
>> the same ietmes happen on OmniOS , the following is a OmniOS-discuss mail
>> .
>>
>> That caused by availrmem not enough for memory locked .
>>
>> Is that mean I can only use about 14G memory for KVM VM in a 32G memory
>> installed server?
>> OS can only locked about 14G memory .
>>
>> ——————FOLLOWING--------------
>>
>> Hello, guys：
>>
>> I has a server running OmniOS v11 r151016. the server have 32G memory .
>> I star tow kvm virtual machines by running the following commands:
>>
>> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:12 -cpu host -smp 4 -m 8192
>> -no-hpe
>>
>>
>> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:11 -cpu host -smp 2 -m 4096
>> -no-hpe
>>
>> one use 8G memory and the other one use 4G memory.
>>
>> now the memory usage of the system as following:
>>
>> root at BLCC01:/root# prtconf | grep Memory
>> Memory size: 32760 Megabytes
>> root at BLCC01:/root# echo "::memstat" | mdb -k
>> Page Summary Pages MB %Tot
>> ------------ ---------------- ---------------- ----
>> Kernel 549618 2146 7%
>> ZFS File Data 668992 2613 8%
>> Anon 3198732 12495 38%
>> Exec and libs 1411 5 0%
>> Page cache 4402 17 0%
>> Free (cachelist) 10578 41 0%
>> Free (freelist) 3950545 15431 47%
>>
>> Total 8384278 32751
>> Physical 8384277 32751
>> root at BLCC01:/root# swap -sh
>> total: 12G allocated + 35M reserved = 12G used, 6.8G available
>> root at BLCC01:/root# swap -l
>> swapfile dev swaplo blocks free
>> /dev/zvol/dsk/rpool/swap 263,2 8 8388600 8388600
>> root at BLCC01:/root#
>>
>>
>> root at BLCC01:/root# prctl $$
>>
>> project.max-locked-memory
>> usage 12.0GB
>> system 16.0EB max deny
>> -
>> project.max-port-ids
>> privileged 8.19K - deny
>> -
>> system 65.5K max deny
>> -
>> project.max-shm-memory
>> privileged 8.00GB - deny
>> -
>> system 16.0EB max deny
>> -
>>
>>
>>
>>
>>
>>
>> #prstat -J
>>
>> PROJID NPROC SWAP RSS MEMORY TIME CPU PROJECT
>> 1 5 12G 12G 38% 1:07:23 5.6% user.root
>> 0 43 72M 76M 0.2% 0:00:59 0.0% system
>> 3 5 4392K 14M 0.0% 0:00:00 0.0% default
>>
>>
>>
>> then I start the third vm (4G memory), it got the following error :
>>
>>
>> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:2 -cpu host -smp 2 -m 4096
>> -no-hpet
>>
>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still
>> trying...
>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still
>> trying...
>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still
>> trying...
>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still
>> trying...
>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still
>> trying...
>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still trying…
>>
>>
>> I got 15G free memory in the system, why qemu-system-x86_64 can not locked
>> enough memory ？
>>
>> Thanks for your help !
>>
>> sorry for my poor english !
>>
>>
>>
>>
>> -----------------------------------
>> fcliang
>>
>>
>>
>> Thank for your help!
>>
>> when the server boot up, it has 7989066 pages availrmem. after I launch
>> one VM (8Gmemory), availrmem decrease to 4756624 .
>>
>>
>>
>> 7989066-4756624 = 3232442
>>
>> 3232442/256 = 12626.7265625 / 1024 = 12.3G
>>
>>
>>
>> root at BLCC01:/root# mdb -ke 'availrmem/D ; pages_pp_maximum/D'
>> availrmem:
>> availrmem: 7989066
>> pages_pp_maximum:
>> pages_pp_maximum: 325044
>>
>> root at BLCC01:/root# qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:12 -cpu
>> host -smp 4 -m 8192 -no-hpe
>>
>>
>>
>> root at BLCC01:/root# mdb -ke 'availrmem/D ; pages_pp_maximum/D'
>> availrmem:
>> availrmem: 4756624
>> pages_pp_maximum:
>> pages_pp_maximum: 325044
>> root at BLCC01:/root#
>>
>>
>> That mean the VM use 12.3G availrmem , how it happens ?
>>
>> Thank !
>>
>>
>>
>>
>> ------------------------------
>> fcliang
>>
>>
>>
>>
>> On Dec 2, 2015, at 1:37, Joshua M. Clulow <josh at sysmgr.org> wrote:
>>
>>> On 1 December 2015 at 09:11, Dan McDonald <danmcd at omniti.com> wrote:
>>>>> On Dec 1, 2015, at 12:03 PM, Fucai.Liang <fcliang at baolict.com> wrote:
>>>>> then I start the third vm (4G memory), it got the following error :
>>>>> qemu-system-x86_64 -enable-kvm -vnc 0.0.0.0:2 -cpu host -smp 2 -m
>> 4096 -no-hpet
>>>>>
>>>>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still
>> trying...
>>>>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still
>> trying...
>>>>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still
>> trying...
>>>>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still
>> trying...
>>>>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still
>> trying...
>>>>> qemu_mlock: have only locked 1940582400 of 4294967296 bytes; still
>> trying…
>>>>>
>>>>> I got 15G free memory in the system, why qemu-system-x86_64 can not
>> locked enough memory ？
>>>> What does "vmstat 1 5" say prior to your launch of the third VM?
>>>
>>> I suspect it will show you have free memory available, but that what
>>> is really happening is getting here:
>>>
>>>
>> https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/vm/seg_vn.c#L7989-L8002
>>>
>>> This is likely failing in page_pp_lock() because "availrmem" has
>>> fallen below "pages_pp_maximum":
>>>
>>>
>> https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/vm/vm_page.c#L3817-L3818
>>>
>>> We set this value here, though it can be overridden in "/etc/system":
>>>
>>>
>> https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/vm/vm_page.c#L423-L436
>>>
>>> You can look at the current values with mdb:
>>>
>>> mdb -ke 'availrmem/D ; pages_pp_maximum/D'
>>>
>>> Increasing this value doesn't seem to be without risk: I believe that
>>> it can lead to memory exhaustion deadlocks, amongst other things. I
>>> don't know if it's expected to be tuneable without a reboot.
>>>
>>>
>>> Cheers.
>>>
>>> --
>>> Joshua M. Clulow
>>> UNIX Admin/Developer
>>> http://blog.sysmgr.org
>>
>> _______________________________________________
>> OmniOS-discuss mailing list
>> OmniOS-discuss at lists.omniti.com
>> http://lists.omniti.com/mailman/listinfo/omnios-discuss
>>
>>
>>
>>
>> ------------------------------
>> Fucai Liang
>>
>>
>>
>>
>> _______________________________________________
>> openindiana-discuss mailing list
>> openindiana-discuss at openindiana.org
>> http://openindiana.org/mailman/listinfo/openindiana-discuss
>>
> _______________________________________________
> openindiana-discuss mailing list
> openindiana-discuss at openindiana.org
> http://openindiana.org/mailman/listinfo/openindiana-discuss
More information about the openindiana-discuss
mailing list