Message Boards Message Boards

17 Replies
19 Total Likes
View groups...
Share this post:

Documentation: manifest file issue

Posted 2 years ago

I have a paclet and I've added the {"Documentation"} extension. I'm using the Document Tools palette to help build it. To make it easy, I've created one Reference Page (there are 300+ function in my Paclet). When running PacletBuild I get an error in Join:

Join::heads: Heads List and Missing at positions 1 and 2 are expected to be the same.

The Call Stack shows:

 Missing["NotAvailable"], <a list of all the files in the Kernel folder>]

What is PacletBuild looking for that I'm missing?

Second error: After the join error, I get an error on my actual reference page:

Throw::nocatch: Uncaught Throw[Illegal manifest file: {Documentation/English/ReferencePages/Guides/Alex.nb,Documentation/English/ReferencePages/Symbols/applyEach.nb}] returned to top level.

The Call Stack doesn't have any other information. Any help is appreciated.

POSTED BY: Andrew Yule
17 Replies

Hi Andrew, thank you for bringing this bug to our attention. I've fixed this bug, and that fix will go out in an upcoming update to PacletTools. The workaround you discovered is a good one.

POSTED BY: Connor Gray

Connor, I work with Andrew. We ended up creating our own PacletExtensionHandler that specifically returns {} as the list of Files for the "Path" extension. This PacletExtensionHandler, which is called a BuildHandler below, also has the option to skip the documentation build, which is very useful while developing and testing the Paclet. It would be a great option to have in PacletBuild.

Our call to PacletBuild:

PacletTools`PacletBuild[$AlexDeveloperPath <> "/Paclet",
      OverwriteTarget -> True,
      PacletExtensionHandlers :> alexBuildHandler

Here's our BuildHandler:

alexBuildHandler := <|
    "Documentation" -> <|"DefaultRoot" -> "Documentation",
      "Files" -> PacletTools`PacletExtensionFiles`Private`handleExt,
      "Build" -> docBuildHandler|>,
    "Path" -> <|"DefaultRoot" -> "Kernel", "Files" -> ({} &), "Build" -> ({} &)|>

The docBuildHandler is defined with the following functions. $AlexDeveloperPath is our developer directory. When it's done building, we save a file "LastDocBuild" which contains the results of PacletBuild``Private``handleExt, which tracks the location of the built document files

defineDocumentHandler[aBuildDocs_] := Block[{},
    docBuildHandler = buildAndSaveDocumentObj
    , (* ELSE - return the old documentation build information *)
    docBuildHandler = documentBuildObj;
documentBuildObj::nobuild = "No documentation build found in `1`. Run alexBuildHandler[\"BuildDocumentation\"->True]";
documentBuildObj = Block[{path},
  If[!DirectoryQ[documentBuildPath], RenameDirectory[savedDocumentsPath, documentBuildPath]];
  path = $AlexDeveloperPath <> "/Paclet/build/LastDocBuild.m";
  If[!FileExistsQ[path], Message[documentBuildObj::nobuild, path]];

buildAndSaveDocumentObj[{"Documentation", ___}, pacletRoot_?DirectoryQ, buildPacletRoot_?DirectoryQ, _?ListQ] := Block[
  result = PacletTools`PacletBuild`Private`handleExt[{"Documentation"}, pacletRoot, buildPacletRoot, {}];
  Put[result, $AlexDeveloperPath <> "/Paclet/build/LastDocBuild.m"];
POSTED BY: Eric Smith

Hi Eric,

That is also a good solution, I'm glad to see that the PacletExtensionHandlers functionality has been of use to you.

Regarding the question of how to avoid wasting time (re-)building the documentation, I hope you'll be glad to hear that I've already implemented support for incremental paclet documentation builds, that again will be available in the next version of PacletTools. With incremental rebuilds, only documentation notebooks that have changed since the last time you built the paclet will be processed again.

POSTED BY: Connor Gray

Connor, I love the new PacletBuild! It works as advertised. We converted everything with zero hassle. I've ditched my hacked paclet builder. The incremental builds are working, and it's much easier to add additional documentation and edits. Nice work!

POSTED BY: Eric Smith

Hi Eric, I'm glad to hear that the new incremental functionality is working well for you! User feedback is great to have. Don't hesitate to get in touch if you have any further problems or suggestions for functionality.

POSTED BY: Connor Gray

Connor, I added a new Tech Note to my documentation and I ran into a bug. It looks like the new templates didn't carry over the "New/Modified/Obsolete/Excised In" cell, and that's causing the build to throw errors.

enter image description here

I opened a Tech Note built in 13.0 and copied that cell over to the Metadata block and I was able to build without errors. Would you add this as a bug report?

POSTED BY: Eric Smith

Eric, I have an existing bug report for this issue, and I've got a fix in progress that I'm intending to send out as a dynamic paclet update by the end of the upcoming work week. If that happens as intended, your system will automatically install the update once it becomes available.

POSTED BY: Connor Gray

Wow! Thanks so much for the speedy patch.

POSTED BY: Eric Smith

Connor were there some more bugs introduced? My "SeeAlso" and "ExamplesInitialization" blocks are no longer rendering correctly. enter image description here

POSTED BY: Eric Smith

Hi Eric, that bug is also fixed in the latest version of PacletTools — which is now live! :)

PacletTools v0.0.12 went out as a dynamic paclet update on Monday evening. You can check if it has already been installed by checking PacletObject["PacletTools"]["Version"].

If you see a version lower than v0.0.12, try doing PacletInstall["PacletTools"] to trigger the check for updates.

POSTED BY: Connor Gray

Just checked. I have v0.0.12 now. Thanks!

POSTED BY: Eric Smith

Hi Eric, I just wanted to give you a heads up that in the upcoming Wolfram v13.2.1 release, PacletBuild[..]'s handling of {"Path"} extensions has changed.

PacletBuild[..] will now generate an error if the paclet has a {"Path"} extension. The fix is to move any files that you want to refer to via the "Path" lookup behavior into a new subdirectory within your paclet (e.g. ./<YourPaclet>/TheFiles/), and then name that subdirectory as {"Path", "Root" -> "TheFiles"}.

This change is to fix the issue described in this Mathematica StackExchange post.

Please let me know if you have any questions or concerns about this change.

POSTED BY: Connor Gray

Connor, Thanks for thinking of me.

Here's my main concern: right now, "Path" provides an easy way to add resources and reference them. I have a structure like this:

  - Documentation
  - Kernel
  - CloudAPIs
    -- DeployFunc1.wl
    -- DeployFunc2.wl
  - Resources
    -- SampleData.wl

I use the extension {"Kernel", "Root" ->"Kernel", "Context" ->{"MyContext"}}, so I can get any of the source code using Get["MyContext`SourceCode`"]. And if anything relies on a resource I can use Get["MyContext/Resources/SampleData.wl"].

I think what you're saying is that I need to gather all my random folders in one place... like this:

  - Documentation
  - Kernel
  - RelativeReference
    -- CloudAPIs
      --- DeployFunc1.wl
      --- DeployFunc2.wl
    -- Resources

And then add {"Path", "Root" -> "RelativeReference"} to my PacletInfo.wl. Is that correct? This way, all subfolders under "Paclet" are associated with an extension.

POSTED BY: Eric Smith

Your understanding sounds correct.

Collecting those files under one folder simply means that they are associated with the "Path" extension via the "Root" declaration, but nothing about how they are looked-up will need to change.

Based on your description, you might might alternatively be interested in using "Asset" extensions, which allow your paclet to expose named assets (which just map to a file or directory within your paclet):

{"Asset", "Root" -> "MyAssets", "Assets" -> {
    {"SomeAssetName" -> "TheData.wl"}

where the paclet directory has the structure:

  - ...
  - MyAssets
      - TheData.wl

which can be looked-up using:

PacletObject["NameOfYourPaclet"]["AssetLocation", "SomeAssetName"]
(* Returns "/path/to/NameOfYourPaclet/Assets/TheData.wl" *)

Let me know if you have further questions or run into any issues.

POSTED BY: Connor Gray

I've switched our "License" extension over to an "Asset" extension. It makes more sense. Thanks for pointing it out! The documentation is easier to browse now; I had been primarily using Todd Gayley's cloud notebook:

I'm slightly worried about some smaller packets we deployed to the cloud and how they'll be impacted when 12.3.1. goes live there, but I guess we'll see. Again, thanks for the heads up about the change.

POSTED BY: Eric Smith

I had been primarily using Todd Gayley's cloud notebook:

Just so you're aware, this content was turned into a tutorial in official documentation:

I'm slightly worried about some smaller packets we deployed to the cloud and how they'll be impacted when 12.3.1. goes live there, but I guess we'll see.

In case my earlier comment was ambiguous: paclets with {"Path"} extensions with the default "Root" -> "." will continue to work in v13.2.1.

The only thing that is changing is that PacletBuild[..] (which has a stricter view of paclets) now doesn't support building paclets with {"Path"} extensions. But already installed / cloud deployed paclets will work the same as before v13.2.1.

I hope that alleviates your worry about those paclets.

Again, thanks for the heads up about the change.

You're welcome! Let me know if there is anything further I can clarify or help with.

POSTED BY: Connor Gray
Posted 2 years ago

I found a workaround. This issue stems from the Paclet extensions. These are the extensions:

"Extensions" ->
      {"Documentation", Language -> "English"},
      {"Kernel", "Root" -> "Kernel", "Context" -> {"Alex`"}}

PacletBuild grabs the documentation files, no files are returned for the "Path" extension, and then the functions are pulled.

In the Paclet directory:


both exist... so why not add /Paclet/Path and leave it blank?

That worked. When it grabs /Paclet/Path it's an empty directory and Join[documentList,{},kernelList] is valid whereas Join[documentList,Missing[_],kernelList] is not

POSTED BY: Andrew Yule
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
or Discard

Group Abstract Group Abstract