diff --git a/contributing.md b/contributing.md index 5761720..8c64e76 100644 --- a/contributing.md +++ b/contributing.md @@ -141,7 +141,7 @@ compiled from our own decompiled code to the contents of an object file extracted from the game. To that end, functions have to be matched up between them. In the best case, corresponding functions in each file will have the same name and be in the same section, at which point objdiff can link them -automatically. Otherwise, one has two click on one of the corresponding +automatically. Otherwise, one has to click on one of the corresponding functions in one pane and the other function in the other pane to tell objdiff to link them. Common cases of this are class methods (the names won't match) and implicitly generated functions, such as exception handling code placed in @@ -150,10 +150,10 @@ and implicitly generated functions, such as exception handling code placed in Clicking on a function that's been linked across both object files shows a diff of the disassembly of both versions of the function, with any differences highlighted. The task at hand is to modify the function in the corresponding -source file such that the match percentage reaches 100%. Depending on how you -configure objdiff, it will rebuild automatically whenever you save a change to -a source file, or you can manually rebuild with the "Build" button at the top -of the right pane. +source file (in the `decompile/src/` directory) such that the match percentage +reaches 100%. Depending on how you configure objdiff, it will rebuild +automatically whenever you save a change to a source file, or you can manually +rebuild with the "Build" button at the top of the right pane. There are no hard instructions to give for writing decompiled code. Use Ghidra's decompilation of the function in the CodeBrowser as a starting point, @@ -167,5 +167,87 @@ disassembly, so it might be worth how they're implemented to learn to recognize them in disassembly and recreate them in C++ code. +Whenever you have some decompiled code that you'd like to contribute to the +repository, commit it to your local copy of the repository and create a merge +request to merge it back into the online copy. -## Contrbuting to Delinking + +## Contributing to Delinking +Getting the JSRF binary delinked is just as important as decompiling the +resulting object files, but takes a bit more investment. The concrete task of +a delinking contributor is to populate `symboltable.tsv` and `objects.csv` in +the `delink/` directory, which together enable consistent delinking of object +files. The former lists symbols at different addresses through the whole +executable, while the latter lists the address ranges that have been identified +as separable objects. Both of these things are figured out by combing over the +whole executable in Ghidra. + + +### Updating `symboltable.tsv` +If you have got a bunch of symbols you'd like to add to `symboltable.tsv`, a +workflow has been devised to generate it from your Ghidra project. Before +regenerating the table, however, make sure that you have all of it symbols +already in your project so that you don't end up deleting any. One option is +to import `symboltable.tsv` into your project with the `ImportSymbolsScript.py` +script as mentioned under "Creating a JSRF Ghidra Project," but be aware that +this will overwrite any names you've assigned to the same symbols. + +Once you're ready to export your symbols, open the symbol table +(`Window > Symbol Table`). Open the symbol filter window (cog button near the +top right), and uncheck everything but "User Defined" under "Symbol Source," +"Data Labels" and "Function Labels" under "Symbol Types," "Use Advanced +Filters," and "Non-Externals" under "Non-Externals." This ensures that you +only export symbols that you've defined and are useful for delinking. + +Now we need to configure the columns that we want to export. Right-click on +one of the colum headers, click "Add/Remove Columns..." to open the "Select +Columns" window, and in it check only "Location," "Name," and "Type." Click +"OK" to close the window and ensure that the column order is "Name," +"Location," "Type" (you can drag the column headers to reorder them if needed). + +Now, to actually export the table, right-click on one of the table cells, click +"Select All," and then right-click again on a cell to select "Export > Export +to CSV..." before selecting where to save your exported symbol table. + +The final step is converting this CSV file to the format expected by +`ImportSymbolsScript.py`. Open a shell in the repository's `delink/` directory +and run `make_symboltable.sh` with the path of your exported CSV as an +argument, and `symboltable.tsv` will be overwritten with a new table containing +your exported symbols. + + +### Updating `objects.csv` +`objects.csv` is a listing of addresses for each object file or group of object +files that we've identified. Each column after the first two corresponds to a +section of the executable, with filled cells indicating an address range +occupied by that object file, empty cells indicating that the object occupies +none of that section, and a `?` indicating an unknown address range or +boundary. The `Object` column gives the path under `decompile/target/` to +extract the object file to if the `Delink?` column is `true`, otherwise it's +just a human-readable label for that row. `delink.sh` parses this file and +uses any rows marked for delinking to produce object files. + +A couple criteria should be fulfilled before marking row in `objects.csv` for +extraction. First, of course, the whole row should be filled with an object +path and with address ranges that we're certain of. Make sure that not just +the `.text` section, but also `.text$x` (exception handling code), `.data`, +`.rdata`, and `.rdata$x` (data pointing to exception-handing code) are included +in the object file if applicable! Address ranges also should not include any +padding before or after data or code. Second, all of the symbols within those +address ranges need to be present in `symboltable.tsv`, else delinking after +only importing those symbols won't arrange the object file's internals +correctly (exception-handling code might be appended onto another function, for +example). Because `symboltable.tsv` should only be populated with symbols that +have been manually defined as per the previous section, this means that you +need to define variable names and labels in Ghidra for everything therein (and +ideally everything referenced externally, as well). + +Once an object is ready for extracting, its `Delink?` column should be set to +`true` and the `objdiff.json` file in the `decompile/` directory should be +updated to include it (give it an entry in the `units` list, modelled after +other existing entries minus the `complete` and `symbol_mappings` fields), plus +a `.cpp` file (and `.hpp` file if suitable) for it should be added for it in +the `decompile/src/` directory. Give the extraction via `delink.sh` a test and +make sure everything's working right for this new object file in objdiff. + +Finally, make a merge request to share your work with us!