wpu.nu

Skillnad mellan versioner av "VariableNaming"

Från wpu.nu

m
m
Rad 7: Rad 7:
  
 
Documents are stored as PDF with a filename of the format:
 
Documents are stored as PDF with a filename of the format:
      [document_md5sum].pdf
+
[document_md5sum].pdf
  
 
Document metadata is saved as a json-file with a filename of the format:
 
Document metadata is saved as a json-file with a filename of the format:
      [document_md5sum].json
+
[document_md5sum].json
  in the same folder as it's corresponding pdf-file.
+
in the same folder as it's corresponding pdf-file.
  
 
Exceptions:
 
Exceptions:
  On wiki import the file is copied to the import folder with
+
On wiki import the file is copied to the import folder with
  a descriptive filename, as that is what the corresponding
+
a descriptive filename, as that is what the corresponding
  wiki pages will be named.
+
wiki pages will be named.
  
  On download the file is named to whatever name the file is given upon
+
On download the file is named to whatever name the file is given upon
  download, as a checksum cannot be calculated until the file is on disk.
+
download, as a checksum cannot be calculated until the file is on disk.
  It is then imediately renamed according to the standard
+
It is then imediately renamed according to the standard
  
  On generation the file is named with a unique name (like parent doc + pages to include)
+
On generation the file is named with a unique name (like parent doc + pages to include)
  When the file has been generated, a checksum is calculated and
+
When the file has been generated, a checksum is calculated and
  the file is renamed.
+
the file is renamed.
  
  
  
 
===ATTRIBUTES===
 
===ATTRIBUTES===
 
+
Some document attributes are reserved and given a defined meaning here.
  Some document attributes are reserved and given a defined meaning here.
+
Other attributes can exists, but should not be passed to other programs
  Other attributes can exists, but should not be passed to other programs
 
  
  CamelCase is used due to it being used in the wiki
+
CamelCase is used due to it being used in the wiki
  
  ====GENERAL====
+
====GENERAL====
  
  Type: [interrogation, promemoria, confrontation, letter, etc]'
+
Type: [interrogation, promemoria, confrontation, letter, etc]'
  Title: A well formatted and descriptive title that will be the base name on the wiki
+
Title: A well formatted and descriptive title that will be the base name on the wiki
  WikiTitle: Same as above but after wiki name formatting and folding
+
WikiTitle: Same as above but after wiki name formatting and folding
  Checksum: MD5 checksum of the corresponding PDF-file
+
Checksum: MD5 checksum of the corresponding PDF-file
  SectionCode: A code refering to the ledger
+
SectionCode: A code refering to the ledger
  InterrogationStart: Date and/or time of interrogation start or generation of document
+
InterrogationStart: Date and/or time of interrogation start or generation of document
  InterrogationEnd: Date and/or time of interrogation end
+
InterrogationEnd: Date and/or time of interrogation end
  Language: sv
+
Language: sv
  Interrogator: Name of interrogator
+
Interrogator: Name of interrogator
  Interrogated: Name of interrogated
+
Interrogated: Name of interrogated
  ShortSummary: A short summary displayed on the wiki
+
ShortSummary: A short summary displayed on the wiki
  Author: Name of person generating a document if not the interrogator
+
Author: Name of person generating a document if not the interrogator
  Progress: Stage of document in wiki-terms (e.g. proofread, waiting for OCR)
+
Progress: Stage of document in wiki-terms (e.g. proofread, waiting for OCR)
  Notes: Notes on the document, also used for OCR errors etc
+
Notes: Notes on the document, also used for OCR errors etc
  Stage: The stage in the import process [OCR, waiting, downloaded, etc]
+
Stage: The stage in the import process [OCR, waiting, downloaded, etc]
  Release: True if doc should be deleted in the next iteration
+
Release: True if doc should be deleted in the next iteration
  
  ====SPIDER====
+
====SPIDER====
  
  FoundAt: Name of the page where the download link was found
+
FoundAt: Name of the page where the download link was found
  FoundAtURL: The URL from which the download link was found (if any)
+
FoundAtURL: The URL from which the download link was found (if any)
  DownloadURL: The URL from which the pdf was downloaded
+
DownloadURL: The URL from which the pdf was downloaded
  DownloadURLText: Text tag from the above URL
+
DownloadURLText: Text tag from the above URL
  DownloadDescription: A description found in association with the download URL
+
DownloadDescription: A description found in association with the download URL
  DownloadFileName: The name of the file as downloaded
+
DownloadFileName: The name of the file as downloaded
  
  ====IMPORT====
+
====IMPORT====
  
  NumberOfPages: Number of pages contained in the corresponding PDF
+
NumberOfPages: Number of pages contained in the corresponding PDF
  SizeHuman: PDF file size in human readable form incl units (str)
+
SizeHuman: PDF file size in human readable form incl units (str)
  SizeBytes: PDF file size in bytes (int)
+
SizeBytes: PDF file size in bytes (int)
  SlackDocShouldImportMsgTS: ID of slack-message asking about import
+
SlackDocShouldImportMsgTS: ID of slack-message asking about import
  SlackDocShouldImportResponseTS: ID of slack-response asking about import
+
SlackDocShouldImportResponseTS: ID of slack-response asking about import
  
  ====DERIVED====
+
====DERIVED====
  
  PDFFilename: <checksum>.pdf
+
PDFFilename: <checksum>.pdf
 
   JSONFilename: <checksum>.json
 
   JSONFilename: <checksum>.json
 
   Directory: <base_directory>/<Stage>/
 
   Directory: <base_directory>/<Stage>/
 
   OCRCacheTestFile: Filename which if present should cause use of cached OCR results
 
   OCRCacheTestFile: Filename which if present should cause use of cached OCR results
  WikiIndex: Index:<WikiTitle>
+
WikiIndex: Index:<WikiTitle>
  
 
===LOG===
 
===LOG===
  list of dict
+
list of dict
  log[<date>] = {text:str,...}
+
log[<date>] = {text:str,...}

Versionen från 6 juni 2021 kl. 19.14


DOCUMENT STANDARD

NAMING

Documents are stored as PDF with a filename of the format: [document_md5sum].pdf

Document metadata is saved as a json-file with a filename of the format: [document_md5sum].json in the same folder as it's corresponding pdf-file.

Exceptions: On wiki import the file is copied to the import folder with a descriptive filename, as that is what the corresponding wiki pages will be named.

On download the file is named to whatever name the file is given upon download, as a checksum cannot be calculated until the file is on disk. It is then imediately renamed according to the standard

On generation the file is named with a unique name (like parent doc + pages to include) When the file has been generated, a checksum is calculated and the file is renamed.


ATTRIBUTES

Some document attributes are reserved and given a defined meaning here. Other attributes can exists, but should not be passed to other programs

CamelCase is used due to it being used in the wiki

GENERAL

Type: [interrogation, promemoria, confrontation, letter, etc]' Title: A well formatted and descriptive title that will be the base name on the wiki WikiTitle: Same as above but after wiki name formatting and folding Checksum: MD5 checksum of the corresponding PDF-file SectionCode: A code refering to the ledger InterrogationStart: Date and/or time of interrogation start or generation of document InterrogationEnd: Date and/or time of interrogation end Language: sv Interrogator: Name of interrogator Interrogated: Name of interrogated ShortSummary: A short summary displayed on the wiki Author: Name of person generating a document if not the interrogator Progress: Stage of document in wiki-terms (e.g. proofread, waiting for OCR) Notes: Notes on the document, also used for OCR errors etc Stage: The stage in the import process [OCR, waiting, downloaded, etc] Release: True if doc should be deleted in the next iteration

SPIDER

FoundAt: Name of the page where the download link was found FoundAtURL: The URL from which the download link was found (if any) DownloadURL: The URL from which the pdf was downloaded DownloadURLText: Text tag from the above URL DownloadDescription: A description found in association with the download URL DownloadFileName: The name of the file as downloaded

IMPORT

NumberOfPages: Number of pages contained in the corresponding PDF SizeHuman: PDF file size in human readable form incl units (str) SizeBytes: PDF file size in bytes (int) SlackDocShouldImportMsgTS: ID of slack-message asking about import SlackDocShouldImportResponseTS: ID of slack-response asking about import

DERIVED

PDFFilename: <checksum>.pdf

 JSONFilename: <checksum>.json
 Directory: <base_directory>/<Stage>/
 OCRCacheTestFile: Filename which if present should cause use of cached OCR results

WikiIndex: Index:<WikiTitle>

LOG

list of dict log[<date>] = {text:str,...}