TAG's comment on MiniApp specs

[QingAn]

Hi, I open an issue to discuss the comments from TAG. @espinr @MichaelWangzitao @Sharonzd

The first comment "Origin model" are related to MiniApp specs as a whole.

The second and third comments I think are more directly related to MiniApp Packaging.

---

For clarity, we'd like to separate the concerns we have in terms of priority.

1. BLOCKER: Origin model
   Looking at https://github.com/w3c/miniapp-lifecycle/blob/main/docs/explainer.md#miniapp-origin the definition of Origin is crucial for following the same-origin policy, to ensure that the page is in a secure context to enable APIs that are only available in such context.

On the Web, it is done cryptographically via TLS and OCSP. In app stores, native applications are using validation done by the app store (like universal link), but this is not a first-hand verification (see the point below about responsabilities of the intermediary).

Note that files contained in the package act as a local cache for content relative to the origin, and as such should be verifiable to ensure that the origin-model is applicable here, even if no HTTP request will ever reach the network but would always be dereferenced to the content of the mini app package.

2. BLOCKER: Package can be modified by the intermediary
   As there is no end-to-end cryptographically guaranteed scheme for validating the integrity of the package compared to what the content distributor originally shipped, the current proposal puts the users at the mercy of the app store to guarantee package integrity. We think this goes against the fundamental principles of the web and is a risk. A zero-trust model, such as what was introduced in signed exchanges would mitigate this.

There are notions about digital signatures in the response, but without knowing exactly when the signing happens by who, there isn't a guarantee that the content is as it came from the origin. If there is strong enforcement to guarantee that the signing is done at the point where it leaves the origin (or an equivalent first-party), this would be a way forward.

3. ZIP
   Just to be clear on this, this is more of an issue with regards to random traversal of the underlying content of a given payload. For smaller payloads, the codec used for packaging is not as critical, but for something as content-heavy as a game (which can easily grow up to hundreds of megabytes), requiring full download (due to the layout of a ZIP file) would result in a less-than-optimal user experience.

We are empathetic to the choice made here, but if there is a possibility for swapping this out to something that is friendlier to low-bandwidth environments (e.g. following the game example - you don't need stage 2 content of a game unless you actually reach stage 2, and there are no guarantees that the user will play that long) as saving on transmission is beneficial to the user. (It also allows for faster loading, even for users on performant networks.)

An alternative design would be to use Service Workers as a delivery mechanism, and keep the packages light - but this comes at a cost of complicating the developer experience. Alternatively one could argue this is doable with fetch(), but these are high-level alternative approaches one could take to move away from a monolithic package which is suboptimal with ZIP.

[espinr]

Thanks @QingAn . We need to solve these issues as soon as possible, I agree.

1) About the origin model

If the MiniApp is served through HTTP, we can guarantee the same-origin policy if we always recommend secure transmission (TLS and OCSP), as the Web resources work. So, when a user agent fetches a MiniApp using the Web, it might preserve the origin model.

If there was no such control (because the user agent cannot control the origin, like if you share a MiniApp through other potential protocols or means), the user agents should show some warning, like when browsers access non-secure environments.

In this case, the URL we fetch represents the origin so that we can solve the problem.

We have also proposed an additional member for a security policy (CSP) (w3c/miniapp-manifest#42) in the manifest (i.e., trusted sources). It could identify the origin for those cases that user agents cannot get from the HTTP exchange.

What if we don't have the MiniApp on the Web?

I agree the package should be verified, even if there are no HTTP requests. Should we recommend a Web-based mechanism to confirm that the package comes from a specific origin?

One potential scenario is that different origins distribute the same MiniApp (like different marketplaces), and the package is stored locally in the device. In this case, the user agent doesn't need to fetch the package because it is already in the local filesystem. Still, the MiniApp is the same.

If we need to preserve the concept of origin, we could specify an optional URI to identify the publisher. This could become the origin of the MiniApp?

It could be a field in the manifest, part of the Content Security Policy.

¨content-security-policy" : 
    {
       "origin": "trusted1.example.org", 
        "default-src": [ "trusted1.example.org", "trusted2.example.org" ], 
    }

Adapting the previous example, we would have something like this:

2) Package can be modified by an intermediary

Yes, this is a challenge. The current packaging specification is flexible to include any signatures (and methods) to preserve the identity of the original developer and publisher. We don't recommend any specific encryption method or algorithm to offer flexibility.

Signed HTTP exchanges might be a good solution, but we cannot rely on them because the user agent may not have access to these HTTP headers. For instance, if we have a MiniApp in the file system and we want to install it into your super app. The HTTP exchanges were made before using a Web Browser; they are not accessible to the MiniApp user agent.

The current model enables something like this: adding several signatures (the user agent may verify) and proof of the integrity of the package.

I may be missing something here. But I think we might cover the challenge (although we don't recommend anything specific, we could do it explicitly).

3) ZIP

ZIP could be less efficient, and a large file might cause poor UX. It was already identified in the earliest analysis and explainer. The choice was made based on current practices, the availability of tools, and the nature of MiniApps (light by definition).

I hope this serves as food for thought and start the discussions with some options on the table. Comments?

[Sharonzd]

(English Version)

@QingAn @espinr @MichaelWangzitao I'd like to express Jia's and my opinions. Welcome comments.

1. Origin model
   MiniApp user agents should be compatible with the Web Origin Model. Currently, when opening a MiniApp, the user agent processes the request URL and locates it to the actual resource in the file system. What we need to do more is to provide developers with a Manifest configuration field for describing their Host. Then the user agent needs to verify whether it is the same origin within the package when processing an HTTP resource request.
   Developers will describe the MiniApp "resource" information in their own manifest.json, including "host", the base path of the resource "base_location," and other information (see below code and flow for more details).
   The user agent determines whether to load local/remote resources according to whether it is the same origin and the matching of the path. Meanwhile, the user agent needs to supplement the implementation of the MiniApp same-origin policy specification, reusing the Web Origin mechanism.

2. Package can be modified by the intermediary
   Developers can apply for certificate issuance from trusted third-party CAs or self-signing certificates and configure the signature verification file path "check_cert_asset" in manifest.json, and configure the signature certificate on the server side. The user agent uses signature fingerprint verification to ensure the security of communication and the authenticity of the package.

3. Zip
   MiniApps support a sub-package mechanism, developers can configure the location field in manifest.json to describe the corresponding relationship between the path and the sub-package for accessing the correct resources.
   In addition, the user agent should also introduce the ZIP progressive loading scheme, gradually unpack and load files during the loading process, and start rendering pages that have already been loaded as soon as possible (such as the homepage).

The complete configuration is as follows.

{
    ...
    "resource": {                                        // the resources of the Mini Program
        "host": "v1-0-0.example.miniapp.com",             // the host where the Mini Program resources are located
        "base_location": "resource/root",                // base path at runtime
        "check_cert_asset": "package/cert_fingerprints",  // resource signature verification file, used to access online resources.  eg: https://v1-1-0.example.miniapp.com/resource/root/package/cert_fingerprints
        "location": {
            "/": {                                       // Path: root routing rule, load remote resources by default. Container access address: https://v1-1-0.example.miniapp.com/resource/root
                "sources": "remote"
            },
            "/some-in-main-pkg-resource": {              // Path: /some-in-main-pkg-resource routing rule, load local resources in the main package. Container address: https://v1-1-0.example.miniapp.com/resource/root/some-in-main-pkg-resource
                "sources": "package/main.zip",           // Online download address for the main package: https://v1-1-0.example.miniapp.com/resource/root/package/main.zip
                "root": "some/buildin/res"              // Relative path to the local installation directory in the main package: some/buildin/res
            },
            "/some-in-sub-pkg-resource": {               // Path: /some-in-sub-pkg-resource routing rule, load local resources in a sub-package. Container address: https://v1-1-0.example.miniapp.com/resource/root/some-in-sub-pkg-resource
                "sources": "package/sub1.zip",           // Online download address for the sub-package: https://v1-1-0.example.miniapp.com/resource/root/package/sub1.zip
                "root": "another/buildin/res"           // Relative path to the local installation directory in the sub-package: another/buildin/res
            }
        }
    },
    ...
}

Resource access process:

(Chinese Version )

1. Origin model
   我们认为 MiniApp user agent 应该兼容实现 Web 的同源策略规范。实际上目前在打开小程序时，user agent 会处理请求地址，将其定位到真实的物理资源。我们需要做更多的是，给开发者提供配置方式，以支持描述其 Host，user agent 在处理资源请求时需要校验其是否同源。
   开发者会在自己小程序的 manifest.json 中描述小程序资源字段信息 resource，其中包括 host、资源的基础路径 base_location 等信息（详见下文）。
   user agent 根据 host 的同源情况、以及 path 的匹配性来决定加载本地/远端资源加载，并且复用 Web 的 Origin 模型补充实现小程序同源机制。

2. Package can be modified by the intermediary
   开发者可向受信任的第三方CA 申请颁发证书，或自颁发签名证书，并在 Manifest.json 配置签名校验文件路径 check_cert_asset（详见下文），并在服务器端配置签名证书。User agent 会校验资源的签名信息是否与开发者站点发布的指纹信息相匹配，来确保通信的安全性以及包的真实性。开发者站点身份由 HTTPS 机制保证

3. Zip
   小程序支持分包机制，开发者可以在 Manifest.json 中配置 location 字段（详见下文），以描述路径和分包的对应关系，以便访问到正确的资源。
   除此之外，user agent 还应该引入 ZIP 渐进式加载方案，在加载过程中逐步解压和加载文件，对于已经加载的部分页面可以尽早开始渲染（如首页）。

完整的配置：

{
    ...
    "resource": {                                        // 用于描述小程序资源
        "host": "v1-0-0.example.
miniapp.com
",            // 用于描述小程序资源所在host
        "base_location": "resource/root",                // 运行时base路径
        "check_cert_asset": "package/cert_fingerprints", // 资源签名校验文件，访问在线资源，容器地址 https://v1-1-0.example.miniapp.com/resource/root/package/cert_fingerprints
        "location": {
            "/": {                                       // path：root 路由规则，默认加载远端资源，容器访问地址 https://v1-1-0.example.miniapp.com/resource/root
                "sources": "remote"
            },
            "/some-in-main-pkg-resource": {              // path：/some-in-main-pkg-resource 路由规则，加载主包中的本地资源，容器地址 https://v1-1-0.example.miniapp.com/resource/root/some-in-main-pkg-resource
                "sources": "package/main.zip"            // 主包在线下载地址：https://v1-1-0.example.miniapp.com/resource/root/package/main.zip
                "root": "some/buildin/res"               // 主包内本地安装目录相对地址：some/buildin/res
            },
            "/some-in-sub-pkg-resource": {               // path：/some-in-sub-pkg-resource 路由规则，加载主包中的本地资源，容器地址 https://v1-1-0.example.miniapp.com/resource/root/some-in-sub-pkg-resource
                "sources": "package/sub1.zip"            // 分包在线下载地址：https://v1-1-0.example.miniapp.com/resource/root/package/sub1.zip
                "root": "another/buildin/res"            // 分包内本地安装目录相对地址：another/buildin/res
            }
        }
    },
    ...
}

资源访问流程：

[espinr]

Thanks @Sharonzd. In general, I agree that this can mitigate the TAG's concerns. Some comments:

1. Origin model: It's something similar to what I proposed at New member for Content Security Policy miniapp-manifest#42 . A new member to specify a CSP (Content Security Policy) mechanism in the manifest. So I think we can create a proposal based on a new manifest member to specify the security restrictions.

2. Tampering. We can have this as an alternative signature method. The certificates may be stored in the package (EPUB does something similar but in a concrete path, so they don't need to specify the path in the manifest). I think it's feasible.

3. ZIP container. This proposal, keeps the same format and enables the user agent to download more packages from the server. This could mitigate the challenge of large ZIP files. I would include this part as optional.