New design to align with 2.16

1e63837a · Vivek Dutta · cd1b4009 · 1e63837a
Unverified Commit 1e63837a authored 1 year ago by Vivek Dutta
--- a/docs/arch/SoftwareModules.md
+++ b/docs/arch/SoftwareModules.md
+# Life Cycle Management with Software Module Daemon
+
+Life Cycle Management of an application is important aspect of firmware development. It provides a dynamic plug and play experience for the application developers.
+
+Software module deamon designed with adherence to the requirements of TR-369( Appendix I) and TR-069 (Appendix VI), it also provides the TR-181 - (v2.16) datamodel support for 'Device.SoftwareModules.' by using bbfdm micro-service model.
+
+## Responsibilities
+
+Software Module Daemon(sw-mod-d aka swmodd) offers life cycle management of the [OCI images](https://github.com/opencontainers/image-spec/blob/main/spec.md).
+
+It has support of below runtime(s):
+ - CRUN [OCI runtime](https://github.com/opencontainers/runtime-spec/blob/main/schema/README.md)
+ - [LXC](https://linuxcontainers.org/lxc/manpages/man5/lxc.container.conf.5.html)
+ 
+ It provides the isolation between host and guest services using linux namespaces with [cgroups](https://docs.kernel.org/admin-guide/cgroup-v1/index.html).
+ 
+ It can provides an isolated Execution Environment(ExecEnv) based on configuration with the support of [overlayfs](https://docs.kernel.org/filesystems/overlayfs.html).
+ 
+ ## Abbreviations
+ 
+ - OCI: [Open Container Initiative](https://opencontainers.org/)
+ - Service Container: An OCI based application containers with all dependency resolved
+ - Host: Device running linux firmware
+ - Guest: Service container running within the Host
+ - LCM: Life Cycle Management of Application containers
+ - SMM: Software Module Management
+ - EEClass: ExecEnvClass or Execution Environment Class
+ - EE: ExecEnv or Execution Environment
+ - DU: DeploymentUnit or Application container image
+ - EU: Application container runtime instance
+ 
+ ## Known Limitations
+ 
+ - It can only support one ExecEnv
+ - Only supports Linux EE
+ - Only supports Service containers
+ - Only support one instance of each DU
+
+## SWMODD UCI
+
+Below is an example swmodd uci
+
+```bash
+config global 'global'
+    option debug '1'
+    option log_level '3'
+    option lan_bridge 'br-lan'
+    option root '/3rdParty/'
+    option filesystem 'ext3'
+
+config execenv 'execenv_1'
+    option enable '1'
+    option name 'lxc'
+    option diskspace '1003'
+    option memory '1234'
+    option cpu '2'
+    option initial_runlevel '1'
+    option current_runlevel '4'
+    option initial_eu_runlevel '1'
+    option restart_reason ''
+    option restart_count '0'
+    option last_restarted '0001-01-01T00:00:00Z'
+    option created_at '0001-01-01T00:00:00Z'
+
+config du_eu_assoc 'finalfate'
+    option uuid '208e38f5-644e-4a27-90c5-d194c89ff894'
+    option duid '_srv_lxc_finalfate'
+    option status 'Installed'
+    option url 'local://crun_template'
+    option description ''
+    option version ''
+    option du_type 'OCIImage'
+    option installed_at '0001-01-01T00:00:00Z'
+    option eu_autostart '1'
+    option eu_run_level '3'
+    option execenv 'lxc'
+```
+
+## Datamodel mappings
+
+TR-181 - v2.16+ for [usp](https://usp-data-models.broadband-forum.org/tr-181-2-16-0-usp.html) and [cwmp](https://cwmp-data-models.broadband-forum.org/tr-181-2-16-0-cwmp.html) brings new possibilities into LCM, which provides clean and clear mappings for application container deployments.
+
+### System capabilities
+
+The first object which defines/list the capabilities of the current system is 'ExecEnvClass'
+
+| Datamodel                                              | Mapping/Intepretation   |
+| :----------------------------------------------------  | :---------------------  |
+| Device.SoftwareModules.ExecEnvClass.{i}.               | List the runtime capabilities of the system, When an EE gets added it inherits the runtime from that instances |
+| Device.SoftwareModules.ExecEnvClass.{i}.DeploymentUnitRef | Reference of DU which result into existence of this EEClass, this shall be empty. |
+| Device.SoftwareModules.ExecEnvClass.{i}.Name | CRUN or LXC |
+| Device.SoftwareModules.ExecEnvClass.{i}.Vendor | iopsys.eu |
+| Device.SoftwareModules.ExecEnvClass.{i}.Version |CRUN or LXC version|
+| Device.SoftwareModules.ExecEnvClass.{i}.Capability.{i}. | This should have one instance per EEClass to segregate between runtime dependencies |
+| Device.SoftwareModules.ExecEnvClass.{i}.Capability.{i}.Specification |OCIImage |
+| Device.SoftwareModules.ExecEnvClass.{i}.Capability.{i}.SpecificationURI |https://github.com/containers/crun/blob/main/crun.1.md OR https://linuxcontainers.org/lxc/|
+| Device.SoftwareModules.ExecEnvClass.{i}.Capability.{i}.SpecificationVersion | 1.0.0 OR lxc-config version|
+
+When `swmodd` starts, it checks the runtime dependencies and based of runtime availability, it populates the EEClass table. One EEClass instance shall maps to each supported runtime tools availability in the system.
+
+If `skopeo` and `umoci` present in the system, it adds an additional Capability of oci image spec to denote that pulling oci images from container registries are also supported.
+
+ex:
+- `swmodd` detects if system has crun and lxc tools available
+- It adds CRUN as one EEClass instance
+- It further add LXC as next EEClass instance
+- It further add below as SpecificationURI
+  - (CRUN tar bundle support) https://github.com/containers/crun/blob/main/crun.1.md
+  - (LXC tar bundle support) https://linuxcontainers.org/lxc/
+  - (Pull from Registry Support if `spokeo` and `umoci` present) https://github.com/opencontainers/image-spec/blob/main/spec.md
+
+
+#### Create execution environment
+
+Execution environment has closed resemblance with a storage which can be used to keep oci images and further can be used in runtime engines.
+
+| Datamodel                                              | Mapping/Interpretation   |
+| :----------------------------------------------------  | :---------------------  |
+| Device.SoftwareModules.ExecEnvClass.{i}.AddExecEnv()   | Adds a new EE |
+| AddExecEnv() => Name                                   | Name of EE maps to directory name for oci images |
+| AddExecEnv() => ParentExecEnv                          | This shall be empty |
+| AddExecEnv() => AvailableRoles                         | USP datamodel ACL, might support in future |
+| AddExecEnv() => Enable                                 | Enables this EE |
+| AddExecEnv() => AllocatedDiskSpace                     | Max allocated space for this EE |
+| AddExecEnv() => AllocatedMemory                        | Max allocated memory for this EE |
+| AddExecEnv() => AllocatedCPUPercent                    | Max allocated CPU for this EE |
+
+
+Here:
+- Name is mandatory and unique parameter which maps to directory inside UCI:{swmod.global.root} directory.
+- To support the 'AllocatedCPUPercent' and 'AllocatedMemory', `swmodd` starts with cgroup configuration. [TODO: Add details]
+- To support 'AllocatedDiskSpace'
+  - `swmodd` helper script will create an dummy image with provided size in UCI:{swmod.global.root} directory with {Name}.img
+  - Format the newly created image with UCI{swmod.global.filesystem} utility
+  - mounts that image to /{Name}
+- In case of AllocatedDiskSpace not used
+  - It bind mounts the UCI:{swmod.global.root}/{Name} to /{Name} path
+
+
+### List execution environment
+
+| Datamodel                                              | Mapping/Intepretation   |
+| :----------------------------------------------------  | :---------------------  |
+| Device.SoftwareModules.ExecEnv.{i}.                    | maps to UCI:{swmodd.execenv} secsions |
+| Device.SoftwareModules.ExecEnv.{i}.AllocatedCPUPercent | maps to UCI:{swmodd.execenv[].cpu} |
+| Device.SoftwareModules.ExecEnv.{i}.AllocatedDiskSpace  | maps to UCI:{swmodd.execenv[].diskspace} |
+| Device.SoftwareModules.ExecEnv.{i}.AllocatedMemory     | maps to UCI:{swmodd.execenv[].memory} |
+| Device.SoftwareModules.ExecEnv.{i}.AvailableCPUPercent | UCI:{swmodd.execenv[].cpu} - CPU consumed swmodd |
+| Device.SoftwareModules.ExecEnv.{i}.AvailableDiskSpace  | Free disk space of /{Name} directory |
+| Device.SoftwareModules.ExecEnv.{i}.AvailableMemory     | UCI:{swmodd.execenv[].cpu} - memory consumed swmodd |
+| Device.SoftwareModules.ExecEnv.{i}.CreatedAt           | maps to UCI:{swmodd.execenv[].created_at} |
+| Device.SoftwareModules.ExecEnv.{i}.CurrentRunLevel     | maps to UCI:{swmodd.execenv[].current_runlevel} |
+| Device.SoftwareModules.ExecEnv.{i}.Enable              | maps to UCI:{swmodd.execenv[].enable} |
+| Device.SoftwareModules.ExecEnv.{i}.ExecEnvClassRef     | Name of the EEClass it belongs |
+| Device.SoftwareModules.ExecEnv.{i}.InitialExecutionUnitRunLevel | maps to UCI:{swmodd.execenv[].initial_eu_runlevel} |
+| Device.SoftwareModules.ExecEnv.{i}.InitialRunLevel     | maps to UCI:{swmodd.execenv[].initial_runlevel} |
+| Device.SoftwareModules.ExecEnv.{i}.LastRestarted       | maps to UCI:{swmodd.execenv[].last_restarted} |
+| Device.SoftwareModules.ExecEnv.{i}.Name                | maps to UCI:{swmodd.execenv[].name} |
+| Device.SoftwareModules.ExecEnv.{i}.ParentExecEnv       | empty |
+| Device.SoftwareModules.ExecEnv.{i}.ProcessorRefList    | empty |
+| Device.SoftwareModules.ExecEnv.{i}.Status              | `swmodd` service status |
+| Device.SoftwareModules.ExecEnv.{i}.Type                | CRUN or LXC |
+| Device.SoftwareModules.ExecEnv.{i}.Vendor              | Iopsys |
+| Device.SoftwareModules.ExecEnv.{i}.Version             | `swmodd` version |
+
+#### List application volumes
+
+Sometimes OCI images brings/requries extenal volumes. [TODO: Add more info]
+
+| Datamodel                                              | Mapping/Intepretation   |
+| :----------------------------------------------------  | :---------------------  |
+| Device.SoftwareModules.ExecEnv.{i}.ApplicationData.{i}. ||
+| Device.SoftwareModules.ExecEnv.{i}.ApplicationData.{i}.AccessPath ||
+| Device.SoftwareModules.ExecEnv.{i}.ApplicationData.{i}.ApplicationUUID ||
+| Device.SoftwareModules.ExecEnv.{i}.ApplicationData.{i}.Capacity||
+| Device.SoftwareModules.ExecEnv.{i}.ApplicationData.{i}.Encrypted||
+| Device.SoftwareModules.ExecEnv.{i}.ApplicationData.{i}.Name||
+| Device.SoftwareModules.ExecEnv.{i}.ApplicationData.{i}.Retain||
+| Device.SoftwareModules.ExecEnv.{i}.ApplicationData.{i}.Utilization||
+
+#### Remove existing volumes from an EE
+
+Device.SoftwareModules.ExecEnv.{i}.ApplicationData.{i}.Remove()
+
+
+### Remove execution environment
+
+EE can be removed with below operate command:
+
+```bash
+Device.SoftwareModules.ExecEnv.{i}.Remove()
+ => Force
+```
+
+- Based of Force parameter, it stops the EU running in this EE
+- unmounts related ApplicationData volumes
+- unmount the /{Name}
+- Do cleanup
+
+### Reseting execution environment
+[TODO: add details on how to handle]
+
+Device.SoftwareModules.ExecEnv.{i}.Reset()
+
+### Restart execution enviroment
+[TODO: add details on how to handle]
+
+Device.SoftwareModules.ExecEnv.{i}.Restart()
+ => Reason
+ => Force
+
+- restarts the EE
+ 
+### Define runlevel for execution enviroment
+
+Device.SoftwareModules.ExecEnv.{i}.SetRunLevel()
+ => RequestedRunLevel
+
+- Stored in uci, used for other operations like restart()
+
+### Update assigned datamodel roles
+[TODO: Future work]
+Device.SoftwareModules.ExecEnv.{i}.ModifyAvailableRoles()
+ => AvailableRoles
+
+### Update execution enviroment settings
+
+Device.SoftwareModules.ExecEnv.{i}.ModifyConstraints()
+ => Force
+ => AllocatedDiskSpace
+ => AllocatedMemory
+ => AllocatedCPUPercent
+
+- For Memory and CPU updates, it will simply be hanled with cgroup options update and restart of swmodd. 
+- For Disk Space update
+  - Unlock the paths
+  - swmodd resize the allocated volume using resizefs
+  - remounts the paths
+
+### EE startup notification
+
+Device.SoftwareModules.ExecEnv.{i}.Restarted!
+ => RestartTime
+ => RestartReason
+
+
+Device.SoftwareModules.InstallDU()
+ => URL
+ => UUID
+ => Username
+ => Password
+ => ExecutionEnvRef
+ => Signature
+ => RequiredRoles
+ => OptionalRoles
+ => AutoRestart.Enable
+ => AutoRestart.RetryMinimumWaitInterval
+ => AutoRestart.RetryMaximumWaitInterval
+ => AutoRestart.RetryIntervalMultiplier
+ => AutoRestart.ResetPeriod
+
+Device.SoftwareModules.DUStateChange!
+
+Device.SoftwareModules.DeploymentUnitNumberOfEntries
+Device.SoftwareModules.DeploymentUnit.{i}.
+
+Device.SoftwareModules.DeploymentUnit.{i}.Uninstall()
+Device.SoftwareModules.DeploymentUnit.{i}.Update()
+
+Device.SoftwareModules.DeploymentUnit.{i}.Alias
+Device.SoftwareModules.DeploymentUnit.{i}.DUID
+Device.SoftwareModules.DeploymentUnit.{i}.Description
+Device.SoftwareModules.DeploymentUnit.{i}.ExecutionEnvRef
+Device.SoftwareModules.DeploymentUnit.{i}.ExecutionUnitList
+Device.SoftwareModules.DeploymentUnit.{i}.Installed
+Device.SoftwareModules.DeploymentUnit.{i}.InternalController
+Device.SoftwareModules.DeploymentUnit.{i}.LastUpdate
+Device.SoftwareModules.DeploymentUnit.{i}.Name
+Device.SoftwareModules.DeploymentUnit.{i}.Resolved
+Device.SoftwareModules.DeploymentUnit.{i}.Status
+Device.SoftwareModules.DeploymentUnit.{i}.URL
+Device.SoftwareModules.DeploymentUnit.{i}.UUID
+Device.SoftwareModules.DeploymentUnit.{i}.Vendor
+Device.SoftwareModules.DeploymentUnit.{i}.VendorConfigList
+Device.SoftwareModules.DeploymentUnit.{i}.VendorLogList
+Device.SoftwareModules.DeploymentUnit.{i}.Version
+
+
+Device.SoftwareModules.ExecutionUnitNumberOfEntries
+Device.SoftwareModules.ExecutionUnit.{i}.
+
+Device.SoftwareModules.ExecutionUnit.{i}.Restart()
+Device.SoftwareModules.ExecutionUnit.{i}.SetRequestedState()
+
+Device.SoftwareModules.ExecutionUnit.{i}.Alias
+Device.SoftwareModules.ExecutionUnit.{i}.AllocatedCPUPercent
+Device.SoftwareModules.ExecutionUnit.{i}.AllocatedDiskSpace
+Device.SoftwareModules.ExecutionUnit.{i}.AllocatedMemory
+Device.SoftwareModules.ExecutionUnit.{i}.ApplicationDataList
+Device.SoftwareModules.ExecutionUnit.{i}.AssociatedProcessList
+Device.SoftwareModules.ExecutionUnit.{i}.AutoStart
+Device.SoftwareModules.ExecutionUnit.{i}.CPUPercentInUse
+Device.SoftwareModules.ExecutionUnit.{i}.Description
+Device.SoftwareModules.ExecutionUnit.{i}.DiskSpaceInUse
+Device.SoftwareModules.ExecutionUnit.{i}.EUID
+Device.SoftwareModules.ExecutionUnit.{i}.ExecEnvLabel
+Device.SoftwareModules.ExecutionUnit.{i}.ExecutionEnvRef
+Device.SoftwareModules.ExecutionUnit.{i}.ExecutionFaultCode
+Device.SoftwareModules.ExecutionUnit.{i}.ExecutionFaultMessage
+Device.SoftwareModules.ExecutionUnit.{i}.MemoryInUse
+Device.SoftwareModules.ExecutionUnit.{i}.Name
+Device.SoftwareModules.ExecutionUnit.{i}.References
+Device.SoftwareModules.ExecutionUnit.{i}.RequestedState
+Device.SoftwareModules.ExecutionUnit.{i}.RunLevel
+Device.SoftwareModules.ExecutionUnit.{i}.Status
+Device.SoftwareModules.ExecutionUnit.{i}.SupportedDataModelList
+Device.SoftwareModules.ExecutionUnit.{i}.Vendor
+Device.SoftwareModules.ExecutionUnit.{i}.VendorConfigList
+Device.SoftwareModules.ExecutionUnit.{i}.VendorLogList
+Device.SoftwareModules.ExecutionUnit.{i}.Version
+
+
+Device.SoftwareModules.ExecutionUnit.{i}.AutoRestart.
+Device.SoftwareModules.ExecutionUnit.{i}.AutoRestart.Enable
+Device.SoftwareModules.ExecutionUnit.{i}.AutoRestart.LastRestarted
+Device.SoftwareModules.ExecutionUnit.{i}.AutoRestart.MaximumRetryCount
+Device.SoftwareModules.ExecutionUnit.{i}.AutoRestart.NextRestart
+Device.SoftwareModules.ExecutionUnit.{i}.AutoRestart.ResetPeriod
+Device.SoftwareModules.ExecutionUnit.{i}.AutoRestart.RetryCount
+Device.SoftwareModules.ExecutionUnit.{i}.AutoRestart.RetryIntervalMultiplier
+Device.SoftwareModules.ExecutionUnit.{i}.AutoRestart.RetryMaximumWaitInterval
+Device.SoftwareModules.ExecutionUnit.{i}.AutoRestart.RetryMinimumWaitInterval
+
+
+Device.SoftwareModules.ExecutionUnit.{i}.HostObjectNumberOfEntries
+Device.SoftwareModules.ExecutionUnit.{i}.HostObject.{i}.
+Device.SoftwareModules.ExecutionUnit.{i}.HostObject.{i}.Alias
+Device.SoftwareModules.ExecutionUnit.{i}.HostObject.{i}.Destination
+Device.SoftwareModules.ExecutionUnit.{i}.HostObject.{i}.Options
+Device.SoftwareModules.ExecutionUnit.{i}.HostObject.{i}.Source
+
+
+
+Device.SoftwareModules.ExecutionUnit.{i}.Extensions.