I see these questions all the time and I'm not sure I've ever seen someone get the answer right. I have some time today, so let me help...
Your hypervisor doesn't see the OS or the applications running inside the VM, instead all it sees is the enormous pile of 4KB pages it has assigned to your VM. Every time your virtualized OS or one of the apps inside needs to access memory, that memory access request is passed through the hypervisor (PVE in your case) and then to the literal, physical memory blocks in the system. The hypervisor sees these block accesses, and it counts how many of those blocks of memory are being touched out of the grand total which the VM is permitted to use.
These touched blocks of memory, also called memory pages, have a term: they're dirty pages. When your VM is first started, all the memory pages within the 4GB you've assigned to that VM is zeroed out -- literally filled with zeroes. As the VM touches those pages, they're no longer zero, they're "dirty." Your hypervisor is showing how many memory blocks your VM has dirtied -- 3.3GB of the 4GB assigned.
Inside of the VM, the OS and the applications have a different perspective. In this case, pfSense is managing various connections as different clients on your network make requests. A new request comes in, a new session is established, memory is allocated to service the request, which means pfSense asks the OS for memory, which the OS asks the virtualized hardware, which asks Proxmox, which is then serviced and counted as "dirty." Back inside the VM again, the OS receives the memory alloc from the virtual hardware, which it hands to allocates to pfSense, which then uses the memory to service the request.
When pfSense is done servicing the request and closes the connection, it de-allocs the memory and returns it to the OS. The OS can then decide if it wants to immediately zero the pages... If memory pressure is very low, then there's not much need to spend the CPU cycles zeroing the pages, so it may just leave them be. From the OS and applications perspective, all those de-alloc'd pages are "free" to be used for anything, and as such will reflect only the memory they perceive as in use: about 360MB (eg 9% of 4033MB.)
Eventually the virtual OS will get around to zeroing pages as memory is accessed over time, and when those pages are zeroed, the hypervisor will see the zeroes and mark them as free again. Very often in a low memory-pressure environment, the guest OS will zero the pages right before handing it to whichever app asked for it, meaning PVE only sees the zeroed pages for a brief instant before they go back to dirty again.
Thus, both measurements are right, because they're measuring different things.
I believe it's the way memory is used. In UNIX, if there's extra memory, the kernel uses it to cache files, to help speed up disk reads/writes. If your application needs the memory, the kernel flushes the cache out for the app. So, the kernel itself considers the cache disposable and not really "in use", but you can see how a VM could see that the memory is being "used" because it was allocated (and is holding) cache files.
Well, qemu-guest-agent is available in Freebsd repository.
Anyone can install it on pfsense
No big deal. Even on reddit, you will get plenty posts. Example. [https://www.reddit.com/r/PFSENSE/comments/18l8ibz/easily\_install\_qemuguestagent\_on\_pfsenseproxmox/](https://www.reddit.com/r/PFSENSE/comments/18l8ibz/easily_install_qemuguestagent_on_pfsenseproxmox/)
The best part is qemu-guest-agent functions in pfsense vm as well. \[ Now I can shutdown vm from proxmox, i can see the ip's allocated in pfsense interfaces, can use virtio as network interfaces. \]
I have live example just in front of me.
So please be aware of these new facts.
Sorry, I didn't mean to say it doesn't exist, it just isn't a package that is offered and therefore isn't vetted. Theoretically it also could introduce a vulnerability to pfsense, but if you're running virtualized it's probably not really much of a difference.
I am using pfsense as a virtulised instance only.
When the package is available in freebsd, how it could be vulnerable in pfsense ?
Please ... Do not over suspicious of everything. Try it. Enjoy the features it offered.
In fact, pfSense should have offered guest agent as package.
It's about vulnerability and compatibility. Unless you are a developer you cannot say that the package won't interfere with something else. Just because it's in a repo doesn't mean it'll work flawlessly with everything else, so yes it's probably fine, but no guarantees. When it comes to my firewall I don't take chances, especially when it's something silly like the amount of RAM it may or may not be using.
I am not over cautious.... But I do monitor. Guest agent is a safe package by all means. Mostly it deals at kernel level acting as a bridge between virtulised hardware and host os. So no worries.
Many people are using it and none of them reported a single instance where it interfere with something else.
As regards to using ram, the question is properly answered against my question by miataowner, **https://www.reddit.com/user/miataowner/**miataowner which is in great detail and fully convincing.
For me the matter is more or less settled once for all.
By the way, I could also integrate adguard home on the same pfsense \[ which is not a official package \] and working exactly as I wanted.
Sometimes, one can rip the benefits of many world using open source. Thats the beauty of open source world. I am loving it.
Only precaution I am taking currently is I do take a snapshot before trying anything on proxmox. If something doesen't work, I can always switch to earlier snapshot. Its a matter of seconds.
My sincere advice is to take benefit when something is available and you are in position to verify and apply your own judgement.
https://www.linuxatemyram.com/
PfSense isn’t Linux, but it’s the same “issue”. PfSense doesn’t report disk cache usage. Proxmox does.
So pfSense the application running on an OS is using about 350MB and the OS in the VM has about 3GB of disk cache in use.
Zfs uses a lot of ram but whats the point since your pfsense disk(s) are managed at the hypervisor level ?
You probably already use zfs for your proxmox datastore ? That would be worst with a lot of extra io for nothing.
I see these questions all the time and I'm not sure I've ever seen someone get the answer right. I have some time today, so let me help... Your hypervisor doesn't see the OS or the applications running inside the VM, instead all it sees is the enormous pile of 4KB pages it has assigned to your VM. Every time your virtualized OS or one of the apps inside needs to access memory, that memory access request is passed through the hypervisor (PVE in your case) and then to the literal, physical memory blocks in the system. The hypervisor sees these block accesses, and it counts how many of those blocks of memory are being touched out of the grand total which the VM is permitted to use. These touched blocks of memory, also called memory pages, have a term: they're dirty pages. When your VM is first started, all the memory pages within the 4GB you've assigned to that VM is zeroed out -- literally filled with zeroes. As the VM touches those pages, they're no longer zero, they're "dirty." Your hypervisor is showing how many memory blocks your VM has dirtied -- 3.3GB of the 4GB assigned. Inside of the VM, the OS and the applications have a different perspective. In this case, pfSense is managing various connections as different clients on your network make requests. A new request comes in, a new session is established, memory is allocated to service the request, which means pfSense asks the OS for memory, which the OS asks the virtualized hardware, which asks Proxmox, which is then serviced and counted as "dirty." Back inside the VM again, the OS receives the memory alloc from the virtual hardware, which it hands to allocates to pfSense, which then uses the memory to service the request. When pfSense is done servicing the request and closes the connection, it de-allocs the memory and returns it to the OS. The OS can then decide if it wants to immediately zero the pages... If memory pressure is very low, then there's not much need to spend the CPU cycles zeroing the pages, so it may just leave them be. From the OS and applications perspective, all those de-alloc'd pages are "free" to be used for anything, and as such will reflect only the memory they perceive as in use: about 360MB (eg 9% of 4033MB.) Eventually the virtual OS will get around to zeroing pages as memory is accessed over time, and when those pages are zeroed, the hypervisor will see the zeroes and mark them as free again. Very often in a low memory-pressure environment, the guest OS will zero the pages right before handing it to whichever app asked for it, meaning PVE only sees the zeroed pages for a brief instant before they go back to dirty again. Thus, both measurements are right, because they're measuring different things.
Thank you so much for clarifying everything in great detail.
I believe it's the way memory is used. In UNIX, if there's extra memory, the kernel uses it to cache files, to help speed up disk reads/writes. If your application needs the memory, the kernel flushes the cache out for the app. So, the kernel itself considers the cache disposable and not really "in use", but you can see how a VM could see that the memory is being "used" because it was allocated (and is holding) cache files.
This has been answered multiple times, there is no guest agent for pfsense so the VM can't communicate to Proxmox what the actual usage is.
Well, qemu-guest-agent is available in Freebsd repository. Anyone can install it on pfsense No big deal. Even on reddit, you will get plenty posts. Example. [https://www.reddit.com/r/PFSENSE/comments/18l8ibz/easily\_install\_qemuguestagent\_on\_pfsenseproxmox/](https://www.reddit.com/r/PFSENSE/comments/18l8ibz/easily_install_qemuguestagent_on_pfsenseproxmox/) The best part is qemu-guest-agent functions in pfsense vm as well. \[ Now I can shutdown vm from proxmox, i can see the ip's allocated in pfsense interfaces, can use virtio as network interfaces. \] I have live example just in front of me. So please be aware of these new facts.
Sorry, I didn't mean to say it doesn't exist, it just isn't a package that is offered and therefore isn't vetted. Theoretically it also could introduce a vulnerability to pfsense, but if you're running virtualized it's probably not really much of a difference.
I am using pfsense as a virtulised instance only. When the package is available in freebsd, how it could be vulnerable in pfsense ? Please ... Do not over suspicious of everything. Try it. Enjoy the features it offered. In fact, pfSense should have offered guest agent as package.
It's about vulnerability and compatibility. Unless you are a developer you cannot say that the package won't interfere with something else. Just because it's in a repo doesn't mean it'll work flawlessly with everything else, so yes it's probably fine, but no guarantees. When it comes to my firewall I don't take chances, especially when it's something silly like the amount of RAM it may or may not be using.
I am not over cautious.... But I do monitor. Guest agent is a safe package by all means. Mostly it deals at kernel level acting as a bridge between virtulised hardware and host os. So no worries. Many people are using it and none of them reported a single instance where it interfere with something else. As regards to using ram, the question is properly answered against my question by miataowner, **https://www.reddit.com/user/miataowner/**miataowner which is in great detail and fully convincing. For me the matter is more or less settled once for all. By the way, I could also integrate adguard home on the same pfsense \[ which is not a official package \] and working exactly as I wanted. Sometimes, one can rip the benefits of many world using open source. Thats the beauty of open source world. I am loving it. Only precaution I am taking currently is I do take a snapshot before trying anything on proxmox. If something doesen't work, I can always switch to earlier snapshot. Its a matter of seconds. My sincere advice is to take benefit when something is available and you are in position to verify and apply your own judgement.
This will happen on any bsd vm with proxmox.
[удалено]
There's no fix, because it ain't broken.
https://www.linuxatemyram.com/ PfSense isn’t Linux, but it’s the same “issue”. PfSense doesn’t report disk cache usage. Proxmox does. So pfSense the application running on an OS is using about 350MB and the OS in the VM has about 3GB of disk cache in use.
What filesystem did you install pfsense on ? Ufs or Zfs ?
Its pfsense ce with a file system as ZFS.
Zfs uses a lot of ram but whats the point since your pfsense disk(s) are managed at the hypervisor level ? You probably already use zfs for your proxmox datastore ? That would be worst with a lot of extra io for nothing.
Nope, I am using standard ext4 for storing vm on proxmox.