[WIP] Add scripts for sparse elf generation#2078
Conversation
abejgonzalez
left a comment
abejgonzalez
left a comment
There was a problem hiding this comment.
Good 1st pass. I think the main thing I want to see is a top-level script that given a ELF file generates the final sparse ELF file. Users shouldn't have to know what sequence to run all the underlying scripts.
| void usage(const char *progname) { | ||
| std::cerr << "Usage: " << progname << " <input_elf_file> <chunk_size_in_bytes> <output_file>" << std::endl; | ||
| } |
There was a problem hiding this comment.
Nit: Move this into main since it's small, no need for an extra function.
There was a problem hiding this comment.
I would also add more of a description here. This processes a RISC-V elf file serially in chunks (specified by the user) and emits a metadata file indicating which chunks (w/ addr + chunk size) are non-zero.
There was a problem hiding this comment.
I would also call the output file, metadata file, in the code + desc.
| std::cout << "\nProcessing Segment " << i << ":" << std::endl; | ||
| std::cout << " Type: " << phdr.p_type << std::endl; | ||
| std::cout << " Offset: 0x" << std::hex << phdr.p_offset << std::dec << std::endl; | ||
| std::cout << " Virtual Address: 0x" << std::hex << phdr.p_vaddr << std::dec << std::endl; | ||
| std::cout << " Physical Address: 0x" << std::hex << phdr.p_paddr << std::dec << std::endl; | ||
| std::cout << " File Size: " << phdr.p_filesz << " bytes" << std::endl; | ||
| std::cout << " Memory Size: " << phdr.p_memsz << " bytes" << std::endl; | ||
| std::cout << " Flags: " << phdr.p_flags << std::endl; |
There was a problem hiding this comment.
How many program headers have you seen in existing ELFs that you've processed. If this number is large I think it would be better to not print (or maybe add a #ifdef DEBUG preprocessor command to disable prints, and have prints disabled by default).
| std::cout << " Segment Virtual Address: 0x" << std::hex << phdr.p_vaddr << std::dec << std::endl; | ||
| std::cout << " Segment Size (filesz): " << phdr.p_filesz << " bytes" << std::endl; | ||
|
|
There was a problem hiding this comment.
Ditto with all debug prints in this loop.
| bool all_zero = std::all_of(segment_data.begin() + chunk_offset, segment_data.begin() + chunk_end, | ||
| [](unsigned char c) { return c == 0; }); |
There was a problem hiding this comment.
Misc. thought, does this get optimized into a check per larger word (i.e. 64b instead of 8b). If not, maybe this can optimized (low priority).
There was a problem hiding this comment.
Where is the script to compile this file?
| # Define a single MEMORY region | ||
| f.write("MEMORY\n") | ||
| f.write("{\n") | ||
| f.write(" MEM (rwx) : ORIGIN = 0x80000000, LENGTH = 0x80000000\n") |
There was a problem hiding this comment.
This length should be auto-generated based on the regions. last_region_addr + chunk - origin_addr.
There was a problem hiding this comment.
The start can also be autogenerated based on the 1st region addr.
| try: | ||
| start = int(start_str, 16) # Start address in hexadecimal | ||
| size = int(size_str, 10) # Size in decimal | ||
| mem_region = "MEM" # Single memory region |
There was a problem hiding this comment.
This might confuse others why there is only a single memory region, also if this is the case (only 1 memory region) no need to have code to assign this to each region, just assume this in the upper function.
There was a problem hiding this comment.
Remove this script since this was just for testing
|
|
||
| def main(): | ||
| if len(sys.argv) != 3: | ||
| print("Usage: python3 extract_regions.py <mem.elf> <regions.txt>") |
There was a problem hiding this comment.
Generally, all scripts should use a shebang. See
chipyard/scripts/build-setup.sh
Line 1 in 73efe72
chipyard/scripts/insert-includes.py
Line 1 in 73efe72
There was a problem hiding this comment.
This script creates the .o files needed for the linker script that's generated... why is this a separate script. I would try to combine the two files together s.t. you do:
- Scan the memory regions to create the metadata file (C++ executable)
- Given the metadata and elf, create all the .o's/etc + linker script
- Pass that output to a automated way to generate the final elf file.
2 participants
SooHyuk: Working on sparse elf file generation to speed up the checkpoint process.
Added some scripts I wrote for elf file scan and linker script generation.