vzctl(8) Containers vzctl(8) NAME vzctl - utility to control an OpenVZ container. SYNOPSIS vzctl [flags] create CTID [--ostemplate name] [--config name] [--private path] [--root path] [--ipadd addr] [--hostname name] vzctl [flags] set CTID parameters [--save] vzctl [flags] destroy | mount | umount | start | stop | restart | status CTID vzctl [flags] exec | exec2 CTID command [arg ...] vzctl [flags] enter CTID [--exec command [arg ...]] vzctl runscript CTID script vzctl --help | --version DESCRIPTION Utility vzctl runs on the host system (otherwise known as Hardware Node, or HN) and performs direct manipulations with containers (CTs). Containers can be referred to by either numeric CTID or by name (see --name option). Note that CT ID <= 100 are reserved for OpenVZ internal purposes. OPTIONS Flags These flags can be used with almost any option. --quiet Disables logging to log file and screen. --verbose Sets logging level to maximum value. Setting container parameters set CTID parameters [--save] [--force] This command sets various container parameters. If a --save flag is given, parameters are saved in container configuration file ctid.conf(5). Use --force to save the parameters even if the current kernel doesn't support OpenVZ. If the container is currently running, vzctl applies these parameters to the container. The following parameters can be used with set command. Miscellaneous --onboot yes|no Sets whether the container will be started during system boot. The container will not be auto-started unless this parameter is set to yes. --bootorder number Sets the boot order priority for this CT. The higher the number is, the earlier in the boot process this container starts. By default this parameter is unset, which is considered to be the lowest priority, so containers with unset bootorder will start last. --root path Sets the path to root directory for this container. This is essentially a mount point for container's root directory. Argument can contain literal string $VEID, which will be substituted with the numeric CT ID. Changing this parameter is not recommended, better edit vz.conf(5) global configuration file. --userpasswd user:password Sets password for the given user in a container, creating the user if it does not exists. Note that this option is not saved in configuration file at all (so --save flag is useless), it is applied to the container (by modifying its /etc/passwd and /etc/shadow files). In case container root filesystem is not mounted, it is automatically mounted, then all the appropriate file changes are applied, then it is unmounted. Note that container should be created before using this option. --disabled yes|no Disable container start. To force the start of a disabled container, use vzctl start --force. --name name Add a name for a container. The name can later be used in subsequent calls to vzctl in place of CTID. --description string Add a textual description for a container. --setmode restart|ignore Whether to restart a container after applying parameters that require the container to be restarted in order to take effect. Networking --ipadd addr Adds IP address to a given container. Note that this option is incremental, so addr are added to already existing ones. --ipdel addr | all Removes IP address addr from a container. If you want to remove all the addresses, use --ipdel all. --hostname name Sets container hostname. vzctl writes it to the appropriate file inside a container (distribution-dependent). --nameserver addr Sets DNS server IP address for a container. If you want to set several nameservers, you should do it at once, so use --nameserver option multiple times in one call to vzctl, as all the name server values set in previous calls to vzctl are overwritten. --searchdomain name Sets DNS search domains for a container. If you want to set several search domains, you should do it at once, so use --searchdomain option multiple times in one call to vzctl, as all the search domain values set in previous calls to vzctl are overwritten. --netif_add ifname[,mac,host_ifname,host_mac,bridge] Adds a virtual Ethernet device (veth) to a given container. Here ifname is the Ethernet device name in the container, mac is its MAC address, host_ifname is the Ethernet device name on the host, and host_mac is its MAC address. MAC addresses should be in the format like XX:XX:XX:XX:XX:XX. bridge is an optional parameter which can be used in custom network start scripts to automatically add the interface to a bridge. All parameters except ifname are optional and are automatically generated if not specified. --netif_del dev_name | all Removes virtual Ethernet device from a container. If you want to remove all devices, use all. Veth interface configuration The following options can be used to reconfigure the already-created virtual Ethernet interface. To select the interface to configure, use --ifname name option. --mac XX:XX:XX:XX:XX:XX MAC address of interface inside a container. --host_ifname name interface name for virtual interface in the host system. --host_mac XX:XX:XX:XX:XX:XX MAC address of interface in the host system. --bridge name Bridge name. Custom network start scripts can use this value to automatically add the interface to a bridge. --mac_filter on|off Enables/disables MAC address filtering for the Container veth device and the possibility of configuring the MAC address of this device from inside the Container. If the filtering is turned on: • the veth device accepts only those packets that have a MAC address in their headers corresponding to that of this device (excluding all broadcast and multicast packets); • it is impossible to modify the veth MAC address from inside the Container. By default, this functionality is enabled for all veth devices existing inside the Container. Resource limits The following options sets barrier and limit for various user beancounters. Each option requires one or two arguments. In case of one argument, vzctl sets barrier and limit to the same value. In case of two colon-separated arguments, the first is a barrier, and the second is a limit. Each argument is either a number, a number with a suffix, or a special value unlimited. Arguments are in items, pages or bytes. Note that page size is architecture-specific, it is 4096 bytes on IA32 platform. You can also specify different suffixes for set parameters (except for the parameters which names start with num). For example, vzctl set CTID --privvmpages 5M:6M should set privvmpages' barrier to 5 megabytes and its limit to 6 megabytes. Available suffixes are: g, G -- gigabytes. m, M -- megabytes. k, K -- kilobytes. p, P -- pages (page is 4096 bytes on x86 architecture, other architectures may differ). You can also specify the literal word unlimited in place of a number. In that case the corresponding value will be set to LONG_MAX, i. e. the maximum possible value. --numproc items[:items] Maximum number of processes and kernel-level threads. Setting the barrier and the limit to different values does not make practical sense. --numtcpsock items[:items] Maximum number of TCP sockets. This parameter limits the number of TCP connections and, thus, the number of clients the server application can handle in parallel. Setting the barrier and the limit to different values does not make practical sense. --numothersock items[:items] Maximum number of non-TCP sockets (local sockets, UDP and other types of sockets). Setting the barrier and the limit to different values does not make practical sense. --vmguarpages pages[:pages] Memory allocation guarantee. This parameter controls how much memory is available to a container. The barrier is the amount of memory that container's applications are guaranteed to be able to allocate. The meaning of the limit is currently unspecified; it should be set to unlimited. --kmemsize bytes[:bytes] Maximum amount of kernel memory used. This parameter is related to --numproc. Each process consumes certain amount of kernel memory - 16 KB at least, 30-50 KB typically. Very large processes may consume a bit more. It is important to have a certain safety gap between the barrier and the limit of this parameter: equal barrier and limit may lead to the situation where the kernel will need to kill container's applications to keep the kmemsize usage under the limit. --tcpsndbuf bytes[:bytes] Maximum size of TCP send buffers. Barrier should be not less than 64 KB, and difference between barrier and limit should be equal to or more than value of numtcpsock multiplied by 2.5 KB. --tcprcvbuf bytes[:bytes] Maximum size of TCP receive buffers. Barrier should be not less than 64 KB, and difference between barrier and limit should be equal to or more than value of numtcpsock multiplied by 2.5 KB. --othersockbuf bytes[:bytes] Maximum size of other (non-TCP) socket send buffers. If container's processes needs to send very large datagrams, the barrier should be set accordingly. Increased limit is necessary for high performance of communications through local (UNIX- domain) sockets. --dgramrcvbuf bytes[:bytes] Maximum size of other (non-TCP) socket receive buffers. If container's processes needs to receive very large datagrams, the barrier should be set accordingly. The difference between the barrier and the limit is not needed. --oomguarpages pages[:pages] Guarantees against OOM kill. Under this beancounter the kernel accounts the total amount of memory and swap space used by the container's processes. The barrier of this parameter is the out-of-memory guarantee. If the oomguarpages usage is below the barrier, processes of this container are guaranteed not to be killed in out-of-memory situations. The meaning of limit is currently unspecified; it should be set to unlimited. --lockedpages pages[:pages] Maximum number of pages acquired by mlock(2). --privvmpages pages[:pages] Allows controlling the amount of memory allocated by the applications. For shared (mapped as MAP_SHARED) pages, each container really using a memory page is charged for the fraction of the page (depending on the number of others using it). For "potentially private" pages (mapped as MAP_PRIVATE), container is charged either for a fraction of the size or for the full size if the allocated address space. In the latter case, the physical pages associated with the allocated address space may be in memory, in swap or not physically allocated yet. The barrier and the limit of this parameter control the upper boundary of the total size of allocated memory. Note that this upper boundary does not guarantee that container will be able to allocate that much memory. The primary mechanism to control memory allocation is the --vmguarpages guarantee. --shmpages pages[:pages] Maximum IPC SHM segment size. Setting the barrier and the limit to different values does not make practical sense. --numfile items[:items] Maximum number of open files. In most cases the barrier and the limit should be set to the same value. Setting the barrier to 0 effectively disables pre-charging optimization for this beancounter in the kernel, which leads to the held value being precise but could slightly degrade file open performance. --numflock items[:items] Maximum number of file locks. Safety gap should be between barrier and limit. --numpty items[:items] Number of pseudo-terminals (PTY). Note that in OpenVZ each container can have not more than 255 PTYs. Setting the barrier and the limit to different values does not make practical sense. --numsiginfo items[:items] Number of siginfo structures. Setting the barrier and the limit to different values does not make practical sense. --dcachesize bytes[:bytes] Maximum size of filesystem-related caches, such as directory entry and inode caches. Exists as a separate parameter to impose a limit causing file operations to sense memory shortage and return an errno to applications, protecting from memory shortages during critical operations that should not fail. Safety gap should be between barrier and limit. --numiptent num[:num] Number of iptables (netfilter) entries. Setting the barrier and the limit to different values does not make practical sense. --physpages pages[:pages] This is currently an accounting-only parameter. It shows the usage of RAM by this container. Barrier should be set to 0, and limit should be set to unlimited. --swappages pages[:pages] The limit, if set, is used to show a total amount of swap space available inside the container. The barrier of this parameter is currently ignored. The default value is unlimited, meaning total swap will be reported as 0. Note that in order for the value to be shown as total swap space, --meminfo parameter should be set to value other than none. CPU fair scheduler parameters These parameters control CPU usage by container. --cpuunits num CPU weight for a container. Argument is positive non-zero number, passed to and used in the kernel fair scheduler. The larger the number is, the more CPU time this container gets. Maximum value is 500000, minimal is 8. Number is relative to weights of all the other running containers. If cpuunits are not specified, default value of 1000 is used. You can set CPU weight for CT0 (host system itself) as well (use vzctl set 0 --cpuunits num). Usually, OpenVZ initscript (/etc/init.d/vz) takes care of setting this. --cpulimit num[%] Limit of CPU usage for the container, in per cent. Note if the computer has 2 CPUs, it has total of 200% CPU time. Default CPU limit is 0 (no CPU limit). --cpus num sets number of CPUs available in the container. Memory output parameters This parameter control output of /proc/meminfo inside a container. --meminfo none No /proc/meminfo virtualization (the same as on host system). --meminfo mode:value Configure total memory output in a container. Reported free memory is evaluated accordingly to the mode being set. Reported swap is evaluated according to the settings of --swappages parameter. You can use the following modes for mode: • pages:value - sets total memory in pages; • privvmpages:value - sets total memory as privvmpages * value. Default is privvmpages:1. Iptables control parameters --iptables name Allow to use the functionality of name iptables module inside the container. To specify multiple names, repeat --iptables for each, or use space-separated list as an argument (enclosed in single or double quotes to protect spaces). The default list of enabled iptables modules is specified by the IPTABLES variable in vz.conf(5). You can use the following values for name: iptable_filter, iptable_mangle, ipt_limit, ipt_multiport, ipt_tos, ipt_TOS, ipt_REJECT, ipt_TCPMSS, ipt_tcpmss, ipt_ttl, ipt_LOG, ipt_length, ip_conntrack, ip_conntrack_ftp, ip_conntrack_irc, ipt_conntrack, ipt_state, ipt_helper, iptable_nat, ip_nat_ftp, ip_nat_irc, ipt_REDIRECT, xt_mac, ipt_recent, ipt_owner. Network devices control parameters --netdev_add name move network device from the host system to a specified container --netdev_del name delete network device from a specified container Disk quota parameters --diskquota yes|no allows to enable or disable disk quota for a container. By default, a global value (DISK_QUOTA) from vz.conf(5) is used. --diskspace num[:num] sets soft and hard disk quota limits, in blocks. First parameter is soft limit, second is hard limit. One block is currently equal to 1Kb. Suffixes G, M, K can also be specified (see Resource limits section for more info on suffixes). --diskinodes num[:num] sets soft and hard disk quota limits, in i-nodes. First parameter is soft limit, second is hard limit. --quotatime seconds sets quota grace period. Container is permitted to exceed its soft limits for the grace period, but once it has expired, the soft limit is enforced as a hard limit. --quotaugidlimit num sets maximum number of user/group IDs in a container for which disk quota inside the container will be accounted. If this value is set to 0, user and group quotas inside the container will not be accounted. Note that if you have previously set value of this parameter to 0, changing it while the container is running will not take effect. Mount option --noatime yes | no Sets noatime flag (do not update inode access times) on filesystem. Capability option --capability capname:on|off Sets a capability for a container. Note that setting capability when the container is running does not take immediate effect; restart the container in order for the changes to take effect. Note a container has default set of capabilities, thus any operation on capabilities is "logical and" with the default capability mask. You can use the following values for capname: chown, dac_override, dac_read_search, fowner, fsetid, kill, setgid, setuid, setpcap, linux_immutable, net_bind_service, net_broadcast, net_admin, net_raw, ipc_lock, ipc_owner, sys_module, sys_rawio, sys_chroot, sys_ptrace, sys_pacct, sys_admin, sys_boot, sys_nice, sys_resource, sys_time, sys_tty_config, mknod, lease, setveid, ve_admin. For detailed description, see capabilities(7). WARNING: setting some of those capabilities may have far reaching security implications, so do not do it unless you know what you are doing. Also note that setting setpcap:on for a container will most probably lead to inability to start it. Device access management --devnodes device:[r][w][q]|none Give the container an access (r - read, w - write, q - disk quota management, none - no access) to a device designated by the special file /dev/device. Device file is created in a container by vzctl. Example: vzctl set 777 --devnodes sdb:rwq. --devices b|c:major:minor|all:[r][w][q]|none Give the container an access to a block or character device designated by its major and minor numbers. Device file have to be created manually. Features management --features name:on|off Enable or disable a specific container feature. Known features are: sysfs, nfs, sit, ipip, ppp, ipgre, bridge, nfsd. Apply config --applyconfig name Read container parameters from the container sample configuration file /etc/vz/conf/ve-name.conf-sample, and apply them, if --save option specified save to the container config file. The following parameters are not changed: HOSTNAME, IP_ADDRESS, OSTEMPLATE, VE_ROOT, and VE_PRIVATE. --applyconfig_map group Apply container config parameters selected by group. Now the only possible value for group is name: to restore container name based on NAME variable in container configuration file. I/O priority management --ioprio priority Assigns I/O priority to container. Priority range is 0-7. The greater priority is, the more time for I/O activity container has. By default each container has priority of 4. Checkpointing and restore Checkpointing is a feature of OpenVZ kernel which allows to save a complete state of a running container, and to restore it later. chkpnt CTID [--dumpfile name] This command saves a complete state of a running container to a dump file, and stops the container. If an option --dumpfile is not set, default dump file name /vz/dump/Dump.CTID is used. restore CTID [--dumpfile name] This command restores a container from the dump file created by the chkpnt command. Performing container actions create CTID [--ostemplate name] [--config name] [--private path] [--root path] [--ipadd addr] [--hostname name] Creates a new container area. This operation should be done once, before the first start of the container. If the --config option is specified, values from example configuration file /etc/vz/conf/ve-name.conf-sample are put into the container configuration file. If this container configuration file already exists, it will be removed. You can use --root path option to sets the path to the mount point for the container root directory (default is VE_ROOT specified in vz.conf(5) file). Argument can contain literal string $VEID, which will be substituted with the numeric CT ID. You can use --private path option to set the path to directory in which all the files and directories specific to this very container are stored (default is VE_PRIVATE specified in vz.conf(5) file). Argument can contain literal string $VEID, which will be substituted with the numeric CT ID. You can use --ipadd addr option to assign an IP address to a container. Note that this option can be used multiple times. You can use --hostname name option to set a host name for a container. destroy | delete Removes a container private area by deleting all files, directories and the configuration file of this container. start [--wait] [--force] Mounts (if necessary) and starts a container. Unless --wait option is specified, vzctl will return immediately; otherwise an attempt to wait till the default runlevel is reached will be made by vzctl. Specify --force if you want to start a container which is disabled (see --disabled). Note that this command can lead to execution of premount, mount and start action scripts (see ACTION SCRIPTS below). stop [--fast] Stops and unmounts a container. Normally, halt(8) is executed inside a container; option --fast makes vzctl use reboot(2) syscall instead which is faster but can lead to unclean container shutdown. Note that this command can lead to execution of stop, umount and postumount action scripts (see ACTION SCRIPTS below). restart Restarts a container, i.e. stops it if it is running, and starts again. Accepts all the start and stop options. Note that this command can lead to execution of some action scripts (see ACTION SCRIPTS below). status Shows a container status. This is a line with five or six words, separated by spaces. First word is literally CTID. Second word is the numeric CT ID. Third word is showing whether this container exists or not, it can be either exist or deleted. Fourth word is showing the status of the container filesystem, it can be either mounted or unmounted. Fifth word shows if the container is running, it can be either running or down. Sixth word, if exists, is suspended. It appears if both a container and its dump file exist (see chkpnt). This command can also be usable from scripts. mount Mounts container private area. Note that this command can lead to execution of premount and mount action scripts (see ACTION SCRIPTS below). umount Unmounts container private area. Note that this command can lead to execution of umount and postumount action scripts (see ACTION SCRIPTS below). Note that stop does umount automatically. exec CTID command Executes command in a container. Environment variables are not set inside the container. Signal handlers may differ from default settings. If command is -, commands are read from stdin. exec2 CTID command The same as exec, but return code is that of command. runscript Run specified shell script in a container, if the container is not running it will be started. enter [--exec command [arg ...]] Enters into a container (giving a container's root shell). This option is a back-door for host root only. The proper way to have CT root shell is to use ssh(1). Option --exec is used to run command with arguments after entering into container. This is useful if command to be run requires a terminal (so vzctl exec can not be used) and for some reason you can not use ssh(1). You need to log out manually from the shell to finish session (even if you specified --exec). Other options --help Prints help message with a brief list of possible options. --version Prints vzctl version. ACTION SCRIPTS vzctl has an ability to execute user-defined scripts when a specific vzctl command is run for a container. The following vzctl commands can trigger execution of action scripts: start, stop, restart, mount and umount. Action scripts are located in the /etc/vz/conf/ directory. There are global and per-CT scripts. Global scripts have a literal prefix of vps. and are executed for all containers. Per-CT scripts have a CTID numeric prefix and are executed for the given container only. There are 8 action scripts currently defined: vps.premount, CTID.premount Global and per-CT mount scripts which are executed for a container before it is mounted. Scripts are executed in the host OS context, while a CT is not yet mounted or running. Global script, if exists, is executed first. vps.mount, CTID.mount Global and per-CT mount scripts which are executed for a container right after it is mounted. Otherwise they are the same as .premount scripts. CTID.start Right after vzctl has started a container, it executes this script in a container context. CTID.stop Right before vzctl has stopped a container, it executes this script in a container context. vps.umount, CTID.umount Global and per-CT umount scripts which are executed for a container before it is unmounted. Scripts are executed in the host OS context, while a CT is mounted. Global script, if exists, is executed first. vps.postumount, CTID.postumount Global and per-CT umount scripts which are executed for a container right after it is unmounted. Otherwise they are the same as .umount scripts. The environment passed to all the *mount scripts is the standard environment of the parent (i.e. vzctl) with two additional variables: $VEID and $VE_CONFFILE. The first one holds the ID of the container, and the second one holds the full path to the container configuration file. If the script needs to get other CT configuration parameters, such as $VE_ROOT, it needs to get those from global and per-CT configuration files. Here is an example of a mount script, which makes host system's /mnt/disk available to container(s). Script name can either be /etc/vz/conf/vps.mount or /etc/vz/conf/CTID.mount. # If one of these files does not exist then something # is really broken [ -f /etc/sysconfig/vz ] || exit 1 [ -f $VE_CONFFILE ] || exit 1 # Source both files. Note the order is important. . /etc/vz/vz.conf . $VE_CONFFILE mount -n --bind /mnt/disk $VE_ROOT/mnt/disk EXIT STATUS Returns 0 upon success, or an appropriate error code in case of an error: 1 Failed to set a UBC parameter 2 Failed to set a fair scheduler parameter 3 Generic system error 5 The running kernel is not an OpenVZ kernel (or some OpenVZ modules are not loaded) 6 Not enough system resources 7 ENV_CREATE ioctl failed 8 Command executed by vzctl exec returned non-zero exit code 9 Container is locked by another vzctl invocation 10 Global OpenVZ configuration file vz.conf(5) not found 11 A vzctl helper script file not found 12 Permission denied 13 Capability setting failed 14 Container configuration file ctid.conf(5) not found 15 Timeout on vzctl exec 16 Error during vzctl chkpnt 17 Error during vzctl restore 18 Error from setluid() syscall 20 Invalid command line parameter 21 Invalid value for command line parameter 22 Container root directory (VE_ROOT) not set 23 Container private directory (VE_PRIVATE) not set 24 Container template directory (TEMPLATE) not set 28 Not all required UBC parameters are set, unable to start container 29 OS template is not specified, unable to create container 31 Container not running 32 Container already running 33 Unable to stop container 34 Unable to add IP address to container 40 Container not mounted 41 Container already mounted 43 Container private area not found 44 Container private area already exists 46 Not enough disk space 47 Bad/broken container (/sbin/init or /bin/sh not found) 48 Unable to create a new container private area 49 Unable to create a new container root area 50 Unable to mount container 51 Unable to unmount container 52 Unable to delete a container 53 Container private area not exist 61 vzquota init failed 62 vzquota on or vzquota setlimit failed 63 Parameter DISKSPACE not set 64 Parameter DISKINODES not set 66 vzquota off failed 67 ugid quota not initialized 71 Incorrect IP address format 74 Error changing password 78 IP address already in use 79 Container action script returned an error 82 Config file copying error 89 IP address not available 91 OS template not found 100 Unable to find container IP address 104 VE_NETDEV ioctl error 105 Container start disabled 106 Unable to set iptables on a running container 107 Distribution-specific configuration file not found 109 Unable to apply a config 129 Unable to set meminfo parameter 130 Error setting veth interface 131 Error setting container name 133 Waiting for container start failed 139 Error saving container configuration file EXAMPLES To create and start "basic" container with ID of 1000 using fedora- core-5 OS template and IP address of 192.168.10.200: vzctl create 1000 --ostemplate fedora-core-5 --config basic vzctl set 1000 --ipadd 192.168.10.200 --save vzctl start 1000 To set number of processes barrier/limit to 80/100, and PTY barrier/limit to 16/20 PTYs: vzctl set 1000 --numproc 80:100 -t 16:20 --save To execute command ls -la in this container: vzctl exec 1000 /bin/ls -la To execute command pipe ls -l / | sort in this container: vzctl exec 1000 'ls -l / | sort' To enter this container and execute command apt-get install vim: vzctl enter 1000 --exec apt-get install vim Note that in the above example you will need to log out from the container's shell after apt-get finishes. To enter this container, execute command apt-get install vim and logout after successful installation (or stay inside the container if installation process failed) use &&: vzctl enter 1000 --exec "apt-get install vim && logout" To enter this container, execute command apt-get install vim and logout independently of exit code of installation process use ;: vzctl enter 1000 --exec "apt-get install vim ; logout" Note that you need to quote the command if you use && or ;. To stop this container: vzctl stop 1000 To permanently remove this container: vzctl destroy 1000 FILES /etc/vz/vz.conf /etc/vz/conf/CTID.conf /etc/vz/conf/vps.{premount,mount,umount,postumount} /etc/vz/conf/CTID.{premount,mount,start,stop,umount,postumount} /proc/vz/veinfo /proc/vz/vzquota /proc/user_beancounters /proc/bc/* /proc/fairsched SEE ALSO vz.conf(5), ctid.conf(5), arpsend(8), vzcalc(8), vzcfgvalidate(8), vzcpucheck(8), vzifup-post(8), vzlist(8), vzmemcheck(8), vzmigrate(8), vzpid(8), vzquota(8), vzsplit(8), http://wiki.openvz.org/UBC. LICENSE Copyright (C) 2000-2010, Parallels, Inc. Licensed under GNU GPL. OpenVZ 16 Nov 2010 vzctl(8)