Skillnad mellan versioner av "VariableNaming"
Från wpu.nu
(Skapade sidan med ' # D O C U M E N T S T A N D A R D # ############### ############### # # N A M I N G # # Documents are stored as PDF with a filename of the format: # [docume...') |
m |
||
| Rad 1: | Rad 1: | ||
| − | + | ==DOCUMENT STANDARD== | |
| − | + | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | + | ===NAMING=== | |
| − | + | ||
| − | + | Documents are stored as PDF with a filename of the format: | |
| − | + | [document_md5sum].pdf | |
| − | + | ||
| − | + | Document metadata is saved as a json-file with a filename of the format: | |
| + | [document_md5sum].json | ||
| + | in the same folder as it's corresponding pdf-file. | ||
| + | |||
| + | Exceptions: | ||
| + | On wiki import the file is copied to the import folder with | ||
| + | a descriptive filename, as that is what the corresponding | ||
| + | wiki pages will be named. | ||
| + | |||
| + | On download the file is named to whatever name the file is given upon | ||
| + | download, as a checksum cannot be calculated until the file is on disk. | ||
| + | It is then imediately renamed according to the standard | ||
| + | |||
| + | On generation the file is named with a unique name (like parent doc + pages to include) | ||
| + | When the file has been generated, a checksum is calculated and | ||
| + | the file is renamed. | ||
| + | |||
| + | |||
| + | |||
| + | ==ATTRIBUTES== | ||
| + | |||
| + | Some document attributes are reserved and given a defined meaning here. | ||
| + | Other attributes can exists, but should not be passed to other programs | ||
| + | |||
| + | CamelCase is used due to it being used in the wiki | ||
| + | |||
| + | ===GENERAL=== | ||
| + | |||
| + | Type: [interrogation, promemoria, confrontation, letter, etc]' | ||
| + | Title: A well formatted and descriptive title that will be the base name on the wiki | ||
| + | WikiTitle: Same as above but after wiki name formatting and folding | ||
| + | Checksum: MD5 checksum of the corresponding PDF-file | ||
| + | SectionCode: A code refering to the ledger | ||
| + | InterrogationStart: Date and/or time of interrogation start or generation of document | ||
| + | InterrogationEnd: Date and/or time of interrogation end | ||
| + | Language: sv | ||
| + | Interrogator: Name of interrogator | ||
| + | Interrogated: Name of interrogated | ||
| + | ShortSummary: A short summary displayed on the wiki | ||
| + | Author: Name of person generating a document if not the interrogator | ||
| + | Progress: Stage of document in wiki-terms (e.g. proofread, waiting for OCR) | ||
| + | Notes: Notes on the document, also used for OCR errors etc | ||
| + | Stage: The stage in the import process [OCR, waiting, downloaded, etc] | ||
| + | Release: True if doc should be deleted in the next iteration | ||
| + | |||
| + | ===SPIDER=== | ||
| + | |||
| + | FoundAt: Name of the page where the download link was found | ||
| + | FoundAtURL: The URL from which the download link was found (if any) | ||
| + | DownloadURL: The URL from which the pdf was downloaded | ||
| + | DownloadURLText: Text tag from the above URL | ||
| + | DownloadDescription: A description found in association with the download URL | ||
| + | DownloadFileName: The name of the file as downloaded | ||
| + | |||
| + | ===IMPORT=== | ||
| + | |||
| + | NumberOfPages: Number of pages contained in the corresponding PDF | ||
| + | SizeHuman: PDF file size in human readable form incl units (str) | ||
| + | SizeBytes: PDF file size in bytes (int) | ||
| + | SlackDocShouldImportMsgTS: ID of slack-message asking about import | ||
| + | SlackDocShouldImportResponseTS: ID of slack-response asking about import | ||
| + | |||
| + | ===DERIVED=== | ||
| + | |||
| + | PDFFilename: <checksum>.pdf | ||
| + | JSONFilename: <checksum>.json | ||
| + | Directory: <base_directory>/<Stage>/ | ||
| + | OCRCacheTestFile: Filename which if present should cause use of cached OCR results | ||
| + | WikiIndex: Index:<WikiTitle> | ||
| + | |||
| + | ==LOG== | ||
| + | list of dict | ||
| + | log[<date>] = {text:str,...} | ||
Versionen från 6 juni 2021 kl. 18.13
DOCUMENT STANDARD
NAMING
Documents are stored as PDF with a filename of the format:
[document_md5sum].pdf
Document metadata is saved as a json-file with a filename of the format:
[document_md5sum].json in the same folder as it's corresponding pdf-file.
Exceptions:
On wiki import the file is copied to the import folder with a descriptive filename, as that is what the corresponding wiki pages will be named.
On download the file is named to whatever name the file is given upon download, as a checksum cannot be calculated until the file is on disk. It is then imediately renamed according to the standard
On generation the file is named with a unique name (like parent doc + pages to include) When the file has been generated, a checksum is calculated and the file is renamed.
ATTRIBUTES
Some document attributes are reserved and given a defined meaning here. Other attributes can exists, but should not be passed to other programs
CamelCase is used due to it being used in the wiki
===GENERAL===
Type: [interrogation, promemoria, confrontation, letter, etc]' Title: A well formatted and descriptive title that will be the base name on the wiki WikiTitle: Same as above but after wiki name formatting and folding Checksum: MD5 checksum of the corresponding PDF-file SectionCode: A code refering to the ledger InterrogationStart: Date and/or time of interrogation start or generation of document InterrogationEnd: Date and/or time of interrogation end Language: sv Interrogator: Name of interrogator Interrogated: Name of interrogated ShortSummary: A short summary displayed on the wiki Author: Name of person generating a document if not the interrogator Progress: Stage of document in wiki-terms (e.g. proofread, waiting for OCR) Notes: Notes on the document, also used for OCR errors etc Stage: The stage in the import process [OCR, waiting, downloaded, etc] Release: True if doc should be deleted in the next iteration
===SPIDER===
FoundAt: Name of the page where the download link was found FoundAtURL: The URL from which the download link was found (if any) DownloadURL: The URL from which the pdf was downloaded DownloadURLText: Text tag from the above URL DownloadDescription: A description found in association with the download URL DownloadFileName: The name of the file as downloaded
===IMPORT===
NumberOfPages: Number of pages contained in the corresponding PDF SizeHuman: PDF file size in human readable form incl units (str) SizeBytes: PDF file size in bytes (int) SlackDocShouldImportMsgTS: ID of slack-message asking about import SlackDocShouldImportResponseTS: ID of slack-response asking about import
===DERIVED===
PDFFilename: <checksum>.pdf JSONFilename: <checksum>.json Directory: <base_directory>/<Stage>/ OCRCacheTestFile: Filename which if present should cause use of cached OCR results WikiIndex: Index:<WikiTitle>
LOG
list of dict
log[<date>] = {text:str,...}