Quentin/DoLuEdu-WordFileParse
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

init: WordFileParse docx→HTML 解析服务（含 Word EQ 域公式修复）

Browse Source 
服务：Spring Boot 3.2 + docx4j 8.3.9，POST /api/v3/server_office_file/file/parse
      接受 .docx，返回 HTML（公式 → LaTeX，图片 → RustFS 外链）

公式支持三类来源：
- OMML（Word 2007+ 公式）      → Python omml_mathml2latex
- MathType OLE 对象            → Ruby mathtype2mathml → MathML → LaTeX
- Word EQ 域（老教辅主力）      → EqToLatexUtil（本次新增）

本次改动：
1. 新增 java-svc/.../core/EqToLatexUtil.java：
   移植自 jby backend-services office-core，EQ 语法 → LaTeX
   （\f→\dfrac、\r→\sqrt、\i→\int/\sum/\prod、\s→^{}/_{}、\b→\left\right、
     \a→array、\x→\boxed/\overline、\o\al 堆叠、圈码、化学符号、极限、连等式）
   品牌标记 JBYU* → WPU*，与采集端一致
2. CtoMathFinder：接上 EQ 分支（原为 not-ported 桩），域代码经
   EqToLatexUtil.convert 转 LaTeX 写入 eqLatexMap（WPEQNUM 占位符）
3. WordParseUtil.dealPlaceholder：EQ 的 LaTeX 同样加 \( \) 包裹
   （原先因 WPEQNUM 例外不加包裹，前端/MathJax 不渲染）

修前症状：教辅 Word 里分式/方程组/上下标整段丢失（如 {−1/3, 1} 变成 {－，1}）
修后实测：46 份教辅 2908 处 EQ 域全部还原，公式总数 2954 处、零未包裹残留

安全：application.yml（含 RustFS 密钥）不入库，提供 application.yml.example；
      OssService 的 @Value 默认密钥已清空
This commit is contained in:
Quentin
2026-09-29 09:43:13 +08:00
commit 3271fee5cc
360 changed files with 86082 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

73
scripts/omml_mathml2latex/data/__init__.py Normal file
View File

@@ -0,0 +1,73 @@
"""
Data module containing symbol mappings, function definitions, and pattern definitions.
This module provides centralized data structures for:
- Unicode to LaTeX symbol mappings
- Mathematical function definitions
- Chemical element definitions
- Common unit definitions
- Regex patterns for text processing
- Operator and symbol set definitions
"""
from .symbols import (
UNICODE_TO_LATEX,
ADDITIONAL_SYMBOL_REPLACEMENTS,
LATEX_COMMAND_REPLACEMENTS,
MATHJAX_UNSUPPORTED_TO_UNICODE,
REGIONAL_SYMBOL_PREFERENCES,
UNICODE_PASSTHROUGH,
)
from .patterns import (
SPACING_CLEANUP_PATTERNS,
DELIMITER_CLEANUP_PATTERNS,
SIMPLE_STRUCTURE_FIX_PATTERNS,
ASSIGNMENT_OPERATOR_REPLACEMENTS,
)
from .functions import (
MATH_FUNCTIONS,
CHEMICAL_ELEMENTS,
)
from .units import COMMON_UNITS
from .operators import (
MATH_SYMBOLS_SET,
SUMMATION_SYMBOLS,
FONT_TO_LATEX,
LARGE_OPERATORS,
LARGE_OPERATORS_UNICODE,
)
from .chemistry_data import (
MULTI_CHAR_ELEMENTS,
SINGLE_CHAR_ELEMENTS,
ALL_CHEMICAL_ELEMENTS,
CHEMICAL_GROUPS,
CHEMISTRY_VARIABLES,
CHINESE_CHEMICAL_TERMS,
)
__all__ = [
"UNICODE_TO_LATEX",
"ADDITIONAL_SYMBOL_REPLACEMENTS",
"LATEX_COMMAND_REPLACEMENTS",
"MATHJAX_UNSUPPORTED_TO_UNICODE",
"REGIONAL_SYMBOL_PREFERENCES",
"UNICODE_PASSTHROUGH",
"SPACING_CLEANUP_PATTERNS",
"DELIMITER_CLEANUP_PATTERNS",
"SIMPLE_STRUCTURE_FIX_PATTERNS",
"ASSIGNMENT_OPERATOR_REPLACEMENTS",
"MATH_FUNCTIONS",
"CHEMICAL_ELEMENTS",
"COMMON_UNITS",
"MATH_SYMBOLS_SET",
"SUMMATION_SYMBOLS",
"FONT_TO_LATEX",
"LARGE_OPERATORS",
"LARGE_OPERATORS_UNICODE",
"MULTI_CHAR_ELEMENTS",
"SINGLE_CHAR_ELEMENTS",
"ALL_CHEMICAL_ELEMENTS",
"CHEMICAL_GROUPS",
"CHEMISTRY_VARIABLES",
"CHINESE_CHEMICAL_TERMS",
]

95
scripts/omml_mathml2latex/data/chemistry_data.py Normal file
View File

@@ -0,0 +1,95 @@
"""
Chemistry-related data for equation formatting.
This module contains chemical elements, groups, and variables used for
detecting and formatting chemistry equations.
"""
# Multi-character chemical elements (2 chars)
# These are STRONG indicators - their presence alone triggers chemistry detection
MULTI_CHAR_ELEMENTS = {
# Period 2
'He', 'Li', 'Be', 'Ne',
# Period 3
'Na', 'Mg', 'Al', 'Si', 'Cl', 'Ar',
# Period 4
'Ca', 'Sc', 'Ti', 'Cr', 'Mn', 'Fe', 'Co', 'Ni', 'Cu', 'Zn',
'Ga', 'Ge', 'As', 'Se', 'Br', 'Kr',
# Period 5
'Rb', 'Sr', 'Zr', 'Nb', 'Mo', 'Tc', 'Ru', 'Rh', 'Pd', 'Ag', 'Cd',
'In', 'Sn', 'Sb', 'Te', 'Xe',
# Period 6
'Cs', 'Ba', 'La', 'Ce', 'Pr', 'Nd', 'Pm', 'Sm', 'Eu', 'Gd', 'Tb', 'Dy',
'Ho', 'Er', 'Tm', 'Yb', 'Lu', 'Hf', 'Ta', 'Re', 'Os', 'Ir', 'Pt',
'Au', 'Hg', 'Tl', 'Pb', 'Bi', 'Po', 'At', 'Rn',
# Period 7
'Fr', 'Ra', 'Ac', 'Th', 'Pa', 'Np', 'Pu', 'Am', 'Cm', 'Bk', 'Cf',
'Es', 'Fm', 'Md', 'No', 'Lr', 'Rf', 'Db', 'Sg', 'Bh', 'Hs', 'Mt', 'Ds',
'Rg', 'Cn', 'Nh', 'Fl', 'Mc', 'Lv', 'Ts', 'Og'
}
# Single-character chemical elements (1 char)
# These need CONTEXT - only formatted as chemistry when chemistry equation is detected
SINGLE_CHAR_ELEMENTS = {
'H', # Hydrogen
'B', # Boron
'C', # Carbon
'N', # Nitrogen
'O', # Oxygen
'F', # Fluorine
'P', # Phosphorus
'S', # Sulfur
'K', # Potassium (but also common variable)
'V', # Vanadium (but also common variable for volume)
'Y', # Yttrium
'I', # Iodine
'W', # Tungsten
'U', # Uranium
}
# All chemical elements (for formatting once chemistry is detected)
ALL_CHEMICAL_ELEMENTS = MULTI_CHAR_ELEMENTS | SINGLE_CHAR_ELEMENTS
# Chemical groups and radicals (strong indicators)
CHEMICAL_GROUPS = {
'OH', # Hydroxide
'SO', # Sulfite/Sulfate (partial)
'NO', # Nitrite/Nitrate (partial)
'CO', # Carbonyl/Carbonate (partial)
'NH', # Amine/Ammonium (partial)
'CH', # Methine/Hydrocarbon (partial)
'PO', # Phosphate (partial)
'ClO', # Chlorate (partial)
}
# Common chemistry variables (should stay italic even in chemistry equations)
# These are NOT chemical elements, they're variables used in chemistry
# NOTE: Keep this list minimal to avoid conflicts with fragmented elements
CHEMISTRY_VARIABLES = {
'c', # Concentration
# NOTE: 'n' removed - conflicts with fragmented Mn (Manganese)
# NOTE: 'M' removed - conflicts with fragmented Mn, Mg (Magnesium)
'p', # Pressure (but P is Phosphorus!)
'T', # Temperature
'm', # Mass
'R', # Gas constant
'E', # Energy (but E is also used in chemistry)
# NOTE: 'V' removed - conflicts with V (Vanadium)
# NOTE: 'K' removed - conflicts with K (Potassium)
# Generic math variables
'x', 'y', 'z',
'a', 'b', 'd', 'q',
}
# Chinese chemical conditions/terms
CHINESE_CHEMICAL_TERMS = {
'高温', # High temperature
'加热', # Heating
'催化剂', # Catalyst
'点燃', # Ignition
'电解', # Electrolysis
'控温', # Temperature control
'加压', # Pressurization
'光照', # Light
'通电', # Electrify
}

71
scripts/omml_mathml2latex/data/functions.py Normal file
View File

@@ -0,0 +1,71 @@
"""
Mathematical functions and chemical elements definitions.
Contains:
- MATH_FUNCTIONS: Set of 60+ recognized mathematical functions
- CHEMICAL_ELEMENTS: Set of 118 chemical elements
"""
# Comprehensive function database
MATH_FUNCTIONS = {
# Trigonometric functions
'sin', 'cos', 'tan', 'sec', 'csc', 'cot',
'sinh', 'cosh', 'tanh', 'sech', 'csch', 'coth',
'arcsin', 'arccos', 'arctan', 'arcsec', 'arccsc', 'arccot',
'asin', 'acos', 'atan', 'asec', 'acsc', 'acot',
'arcsinh', 'arccosh', 'arctanh', 'arcsech', 'arccsch', 'arccoth',
'asinh', 'acosh', 'atanh', 'asech', 'acsch', 'acoth',
# Logarithmic and exponential
'log', 'ln', 'lg', 'exp', 'log2', 'log10',
# Limits and supremum/infimum
'lim', 'limsup', 'liminf', 'sup', 'inf', 'max', 'min',
# Linear algebra
'det', 'dim', 'ker', 'rank', 'trace', 'tr', 'span', 'null',
'col', 'row', 'diag', 'adj',
# Number theory
'gcd', 'lcm', 'mod', 'rem',
# Probability and statistics
'Pr', 'Var', 'Cov', 'Corr', 'Std', 'Med',
# Calculus and analysis
'grad', 'div', 'curl', 'rot', 'arg', 'Im', 'Re',
'sgn', 'sign', 'abs', 'floor', 'ceil', 'round',
# Special functions
'erf', 'erfc', 'gamma', 'Gamma', 'zeta', 'beta', 'Beta',
'sinc', 'jinc',
# Hyperbolic and inverse hyperbolic
'arsinh', 'arcosh', 'artanh', 'arsech', 'arcsch', 'arcoth',
# Other common functions
'deg', 'rad', 'hom', 'end', 'aut', 'id',
}
# Chemical elements for special handling
CHEMICAL_ELEMENTS = {
'H', 'He', 'Li', 'Be', 'B', 'C', 'N', 'O', 'F', 'Ne',
'Na', 'Mg', 'Al', 'Si', 'P', 'S', 'Cl', 'Ar', 'K', 'Ca',
'Sc', 'Ti', 'V', 'Cr', 'Mn', 'Fe', 'Co', 'Ni', 'Cu', 'Zn',
'Ga', 'Ge', 'As', 'Se', 'Br', 'Kr', 'Rb', 'Sr', 'Y', 'Zr',
'Nb', 'Mo', 'Tc', 'Ru', 'Rh', 'Pd', 'Ag', 'Cd', 'In', 'Sn',
'Sb', 'Te', 'I', 'Xe', 'Cs', 'Ba', 'La', 'Ce', 'Pr', 'Nd',
'Pm', 'Sm', 'Eu', 'Gd', 'Tb', 'Dy', 'Ho', 'Er', 'Tm', 'Yb',
'Lu', 'Hf', 'Ta', 'W', 'Re', 'Os', 'Ir', 'Pt', 'Au', 'Hg',
'Tl', 'Pb', 'Bi', 'Po', 'At', 'Rn', 'Fr', 'Ra', 'Ac', 'Th',
'Pa', 'U', 'Np', 'Pu', 'Am', 'Cm', 'Bk', 'Cf', 'Es', 'Fm',
'Md', 'No', 'Lr', 'Rf', 'Db', 'Sg', 'Bh', 'Hs', 'Mt', 'Ds',
'Rg', 'Cn', 'Nh', 'Fl', 'Mc', 'Lv', 'Ts', 'Og'
}
# Mathematical set notation symbols that should be wrapped in \mathrm{}
MATHEMATICAL_SETS = {
'N', 'Z', 'Q', 'R', 'C', # Number sets (Natural, Integer, Rational, Real, Complex)
'A', 'B', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M',
'O', 'P', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', # Generic sets
}

65
scripts/omml_mathml2latex/data/operators.py Normal file
View File

@@ -0,0 +1,65 @@
"""
Operator and symbol set definitions for MathML to LaTeX conversion.
This module provides:
- Mathematical symbol sets for symbol detection
- Summation symbol mappings
- Font to LaTeX command mappings
- Large operator definitions (Unicode versions)
"""
# Mathematical symbol set for symbol detection
MATH_SYMBOLS_SET = {
"↑", "↓", "→", "←", "↔", "×", "⋅", "·", "÷", "∕", "−", # Added U+2212 MINUS SIGN
"∩", "∪", "⊂", "⊃", "⊆", "⊇", "∈", "∉", "∋", "∌",
"≤", "≥", "≠", "≈", "≅", "≡", "≢", "∼", "≃", "≄",
"≜", "≝", "≞", "≟", "≦", "≧", "≨", "≩", "≲", "≳", "≴", "≵", "≶", "≷", "≸", "≹",
"≺", "≻", "⪯", "⪰", "∝", "⊥",
"∥", "∦", "±", "∓", "∗", "∘", "∙", "√", "∞",
"∂", "∇", "∆", "∫", "∬", "∭", "∮", "∯", "∰",
"∑", "∏", "∐", "⋃", "⋂", "⋁", "⋀", "⨁", "⨂", "⨄",
"∀", "∃", "∄", "⊢", "⊣", "⊤", "⊥", "¬", "∧", "∨",
"⟨", "⟩", "⌊", "⌋", "⌈", "⌉", "⏞", "⏟", "∠", "∁"
}
# Summation symbols for specific processing
SUMMATION_SYMBOLS = {
'∑': 'sum',
'∏': 'prod',
'∫': 'integral',
'Σ': 'sum',
'Π': 'prod'
}
# Font family to LaTeX command mapping
FONT_TO_LATEX = {
'Times New Roman': 'mathrm', # Roman/upright text
'Times': 'mathrm', # Times font variant
'Cambria Math': None, # Default math font (no special formatting)
'Cambria': None, # Cambria variant
'Arial': 'mathsf', # Sans-serif
'Helvetica': 'mathsf', # Helvetica sans-serif
'Courier New': 'mathtt', # Typewriter/monospace
'Courier': 'mathtt', # Courier variant
'Computer Modern': None, # LaTeX default math font
'Latin Modern Math': None, # Modern LaTeX math font
}
# Large operators that should use \limits in display mode (LaTeX commands)
LARGE_OPERATORS = [
"\\sum", "\\prod", "\\int", "\\iint", "\\iiint", "\\oint",
"\\bigcup", "\\bigcap", "\\bigvee", "\\bigwedge",
"\\bigoplus", "\\bigotimes", "\\biguplus", "\\coprod",
"\\lim", "\\limsup", "\\liminf", "\\sup", "\\inf",
"\\max", "\\min"
]
# Unicode large operators for detection
LARGE_OPERATORS_UNICODE = [
"∑", "Σ", # Summation
"∏", "Π", # Product
"∫", # Integral
"⋃", "⋂", # Union, Intersection
"⋁", "⋀", # Logical OR, AND
"⨁", "⨂", "⨄", # Direct sum, tensor product, union
]

168
scripts/omml_mathml2latex/data/patterns.py Normal file
View File

@@ -0,0 +1,168 @@
"""
Pattern-based replacements and regex patterns for LaTeX processing.
This module provides regex patterns for:
- Spacing cleanup and normalization
- Delimiter marker cleanup
- LaTeX structure fixes
- Assignment operator normalization
"""
# Assignment operator normalization (line 5529-5530)
ASSIGNMENT_OPERATOR_REPLACEMENTS = [
('∶ =', ':='), # U+2236 RATIO with space and equals
(': =', ':='), # Colon with space and equals
]
# ========================================================================
# Pattern-Based Replacements (Regex Patterns)
# ========================================================================
# Spacing cleanup patterns (consolidated from lines 1289, 1292-1293, 1296, 5566-5567)
SPACING_CLEANUP_PATTERNS = [
# Reduce excessive spaces (line 1289)
(r' {3,}', ' ', 'Reduce 3+ spaces to double space'),
# Remove spaces before delimiters (lines 1292-1293)
(r'\\left\s+\\{', r'\\left\\{', 'Remove space before left brace'),
(r'\\right\s+\\}', r'\\right\\}', 'Remove space before right brace'),
# Remove space between command and braces (line 1296)
(r'\\([a-zA-Z]+)\s+\{', r'\\\1{', 'Remove space between command and opening brace'),
# Add space after set operations when not followed by braces (lines 5566-5567)
(r'\\cap(?![{])', r'\\cap ', 'Add space after \\cap'),
(r'\\cup(?![{])', r'\\cup ', 'Add space after \\cup'),
]
# Delimiter marker cleanup patterns (consolidated from lines 1491-1502, 5433-5447)
# These clean up __DELIM__ markers left from OMML processing
DELIMITER_CLEANUP_PATTERNS = [
# Paired delimiters with content (line 5433)
(r'__DELIM__(.+?)__DELIM__', r'\1', 'Remove paired __DELIM__ markers'),
# Corrupted paired delimiters (lines 5436-5438)
(r'__DELIM_(.+?)__DELIM_', r'\1', 'Remove corrupted paired markers (single underscore)'),
(r'__DELIM__(.+?)__DELIM_', r'\1', 'Remove mixed paired markers (full+corrupted)'),
(r'__DELIM_(.+?)__DELIM__', r'\1', 'Remove mixed paired markers (corrupted+full)'),
# Orphaned markers with single character (lines 5441-5444)
(r'__DELIM__(.)', r'\1', 'Remove orphaned opening marker with character'),
(r'(.)__DELIM__', r'\1', 'Remove orphaned closing marker with character'),
(r'__DELIM_(.)', r'\1', 'Remove corrupted opening marker with character'),
(r'(.)__DELIM_', r'\1', 'Remove corrupted closing marker with character'),
# Standalone markers (lines 5445-5447)
(r'__DELIM__', '', 'Remove any remaining full markers'),
(r'__DELIM_', '', 'Remove any remaining corrupted markers'),
(r'_DELIM__', '', 'Remove reverse corrupted markers'),
# Specific delimiter characters (lines 1491-1497)
(r'__DELIM[_]*\([_]*', '(', 'Clean opening paren markers'),
(r'[_]*\)__DELIM[_]*', ')', 'Clean closing paren markers'),
(r'__DELIM[_]*\[[_]*', '[', 'Clean opening bracket markers'),
(r'[_]*\]__DELIM[_]*', ']', 'Clean closing bracket markers'),
(r'__DELIM[_]*\{[_]*', '{', 'Clean opening brace markers'),
(r'[_]*\}__DELIM[_]*', '}', 'Clean closing brace markers'),
(r'__DELIM[_]*\|[_]*', '|', 'Clean pipe markers (absolute value)'),
# Fragments (lines 1500-1502)
(r'LIM__', '', 'Remove LIM__ fragment'),
(r'LIM_', '', 'Remove LIM_ fragment'),
]
# Simple structure fix patterns (these don't need complex logic)
SIMPLE_STRUCTURE_FIX_PATTERNS = [
# Convert \overset{\dot}{X} to \dot{X} (lines 5570-5571)
(r'\\overset\{\\dot\}\{([a-zA-Z0-9])\}', r'\\dot{\1}', 'Simplify dot overset to \\dot command'),
(r'\\overset\{\.\}\{([a-zA-Z0-9])\}', r'\\dot{\1}', 'Simplify literal dot overset to \\dot command'),
# Fix array column specification (line 5574)
(r'\\begin\{array\{([^}]+)\}\}', r'\\begin{array}{\1}', 'Fix array column specification braces'),
# Convert incorrect fraction to reaction arrow (line 5579)
(r'\\dfrac\{\\underline\{([^}]+)\}\}\{\}', r'\\xlongequal[\\quad]{\1}', 'Convert underline fraction to reaction arrow'),
# Fix nested underset with underbrace (lines 5744-5747)
(r'\\underset\{([^{}]+)\}\{\\underset\{\\underbrace\}\{([^{}]+)\}\}', r'\\underbrace{\2}_{\1}', 'Fix nested underset underbrace pattern'),
# Fix overset with \hat over numbers to \overline (lines 5753-5756)
(r'\\overset\{\\hat\}\{(\\mathrm\{)?(\d+)\}?\}', r'\\overline{\2}', 'Convert hat overset to overline for numbers'),
# Fix underset with textemdash patterns (lines 5802-5823)
(r'\\underset\{\\textemdash\}\{\\underset\{\\textemdash\}\{([^{}]+)\}\}', r'\\underline{\\underline{\1}}', 'Convert nested textemdash underset to double underline'),
(r'\\underset\{\\textemdash\}\{\\underline\{([^{}]+)\}\}', r'\\underline{\\underline{\1}}', 'Convert textemdash underset with underline to double underline'),
(r'\\underset\{\\textemdash\}\{([^{}]+)\}', r'\\underline{\1}', 'Convert textemdash underset to underline'),
]
# ========================================================================
# Additional Symbol Classification Data
# ========================================================================
# Mathematical symbols for classification
MATH_SYMBOLS_SET = {
"↑", "↓", "→", "←", "↔", "×", "⋅", "·", "÷", "∕", "−", # Added U+2212 MINUS SIGN
"∩", "∪", "⊂", "⊃", "⊆", "⊇", "∈", "∉", "∋", "∌",
"≤", "≥", "≠", "≈", "≅", "≡", "≢", "∼", "≃", "≄",
"≜", "≝", "≞", "≟", "≦", "≧", "≨", "≩", "≲", "≳", "≴", "≵", "≶", "≷", "≸", "≹",
"≺", "≻", "⪯", "⪰", "∝", "⊥",
"∥", "∦", "±", "∓", "∗", "∘", "∙", "√", "∞",
"∂", "∇", "∆", "∫", "∬", "∭", "∮", "∯", "∰",
"∑", "∏", "∐", "⋃", "⋂", "⋁", "⋀", "⨁", "⨂", "⨄",
"∀", "∃", "∄", "⊢", "⊣", "⊤", "⊥", "¬", "∧", "∨",
"⟨", "⟩", "⌊", "⌋", "⌈", "⌉", "⏞", "⏟", "∠", "∁"
}
# Summation symbols for specific processing
SUMMATION_SYMBOLS = {
'∑': 'sum',
'∏': 'prod',
'∫': 'integral',
'Σ': 'sum',
'Π': 'prod'
}
# Font family to LaTeX command mapping
FONT_TO_LATEX = {
'Times New Roman': 'mathrm', # Roman/upright text
'Times': 'mathrm', # Times font variant
'Cambria Math': None, # Default math font (no special formatting)
'Cambria': None, # Cambria variant
'Arial': 'mathsf', # Sans-serif
'Helvetica': 'mathsf', # Helvetica sans-serif
'Courier New': 'mathtt', # Typewriter/monospace
'Courier': 'mathtt', # Courier variant
'Computer Modern': None, # LaTeX default math font
'Latin Modern Math': None, # Modern LaTeX math font
}
# Large operators that should use \limits in display mode
LARGE_OPERATORS = [
"\\sum", "\\prod", "\\int", "\\iint", "\\iiint", "\\oint",
"\\bigcup", "\\bigcap", "\\bigvee", "\\bigwedge",
"\\bigoplus", "\\bigotimes", "\\biguplus", "\\coprod",
"\\lim", "\\limsup", "\\liminf", "\\sup", "\\inf",
"\\max", "\\min"
]
# Unicode large operators for detection
LARGE_OPERATORS_UNICODE = [
"∑", "Σ", # Summation
"∏", "Π", # Product
"∫", # Integral
"∮", # Contour integral
"∬", # Double integral
"∭", # Triple integral
"⋂", # Intersection
"⋃", # Union
"⋁", # Logical OR
"⋀", # Logical AND
"⊕", # Direct sum
"⊗", # Tensor product
"⊎", # Multiset sum
"⨁", # Large direct sum
"⨂", # Large tensor product
"⨃", # Large union with plus
"⨆", # Large square union
"⨄", # Large union with plus
]

689
scripts/omml_mathml2latex/data/symbols.py Normal file
View File

@@ -0,0 +1,689 @@
"""
Unicode to LaTeX symbol mappings for OMML/MathML to LaTeX conversion.
This module provides comprehensive Unicode symbol to LaTeX command mappings
for mathematical notation, chemical symbols, and special characters.
Architecture
============
This module defines three primary dictionaries that work together to handle
symbol conversion with MathJax compatibility:
1. UNICODE_TO_LATEX (dict[str, str])
- Maps Unicode symbols → LaTeX commands
- Used for symbols that SHOULD be converted to LaTeX
- Contains ~430 symbol mappings organized by category:
* Greek letters (lowercase and uppercase)
* Mathematical operators and relations
* Arrows and directional symbols
* Set theory and logic symbols
* Calculus and analysis symbols (basic integrals)
* Chemistry symbols and arrows
* Geometric shapes and special characters
2. UNICODE_PASSTHROUGH (set[str])
- Unicode symbols that should NEVER be converted to LaTeX
- These symbols are kept as-is for MathJax compatibility
- Contains ~64 advanced mathematical symbols whose LaTeX commands
are not supported or render poorly in MathJax
- Examples: advanced integrals (∯, ∰), special operators (⊛, ⊚)
- IMPORTANT: Symbols in this set are NOT in UNICODE_TO_LATEX
3. MATHJAX_UNSUPPORTED_TO_UNICODE (dict[str, str])
- Inverse of UNICODE_PASSTHROUGH: LaTeX command → Unicode
- Used in post-processing to convert unsupported LaTeX back to Unicode
- Should be exact inverse of symbols in UNICODE_PASSTHROUGH
Conversion Flow
===============
Input Unicode Symbol
↓
Is it in UNICODE_PASSTHROUGH?
├─ YES → Keep as Unicode (don't convert)
└─ NO → Look up in UNICODE_TO_LATEX
↓
Convert to LaTeX command
↓
Post-process: If command in MATHJAX_UNSUPPORTED_TO_UNICODE
↓
Convert back to Unicode for MathJax rendering
Design Principles
=================
1. **No Overlap**: Symbols should be in EITHER UNICODE_TO_LATEX OR
UNICODE_PASSTHROUGH, never both.
2. **Consistency**: MATHJAX_UNSUPPORTED_TO_UNICODE values must exactly
match UNICODE_PASSTHROUGH contents (they are inverses).
3. **Inverse Mapping**: For any symbol in UNICODE_PASSTHROUGH, if there
exists a LaTeX command in MATHJAX_UNSUPPORTED_TO_UNICODE, the command
names must be consistent.
4. **Special Patterns**:
- Empty braces ({}) only for radical symbols (√, ∛, ∜)
- Placeholders (__DDOT__, __DDDOT__) only for accent detection
- Regional preferences handled separately (see REGIONAL_SYMBOL_PREFERENCES)
Testing
=======
Symbol dictionary integrity is validated by:
- tests/test_symbol_dictionaries.py
This test suite ensures:
- No duplicate keys in any dictionary
- No overlap between UNICODE_TO_LATEX and UNICODE_PASSTHROUGH
- Consistency between MATHJAX_UNSUPPORTED and UNICODE_PASSTHROUGH
- Valid mappings and inverse consistency
"""
# Centralized symbol mappings to eliminate duplication
UNICODE_TO_LATEX = {
# Greek letters (lowercase)
"α": "\\alpha", "β": "\\beta", "γ": "\\gamma", "δ": "\\delta",
"ε": "\\varepsilon", "ζ": "\\zeta", "η": "\\eta", "θ": "\\theta",
"ι": "\\iota", "κ": "\\kappa", "λ": "\\lambda", "μ": "\\mu",
"ν": "\\nu", "ξ": "\\xi", "ο": "\\omicron", "π": "\\pi",
"ρ": "\\rho", "σ": "\\sigma", "τ": "\\tau", "υ": "\\upsilon",
"φ": "\\varphi", "χ": "\\chi", "ψ": "\\psi", "ω": "\\omega",
"ϵ": "\\epsilon", "ϑ": "\\vartheta", "ϕ": "\\phi", "ϱ": "\\varrho",
"ϖ": "\\varpi", "ς": "\\varsigma", "ϰ": "\\varkappa",
# Greek letters (uppercase)
"Α": "A", "Β": "B", "Γ": "\\Gamma", "Δ": "\\Delta",
"Ε": "E", "Ζ": "Z", "Η": "H", "Θ": "\\Theta",
"Ι": "I", "Κ": "K", "Λ": "\\Lambda", "Μ": "M",
"Ν": "N", "Ξ": "\\Xi", "Ο": "O", "Π": "\\Pi",
"Ρ": "P", "Σ": "\\Sigma", "Τ": "T", "Υ": "\\Upsilon",
"Φ": "\\Phi", "Χ": "X", "Ψ": "\\Psi", "Ω": "\\Omega",
# Mathematical constants and special symbols
"∞": "\\infty", "∅": "\\emptyset", "∇": "\\nabla", "∂": "\\partial",
"ℏ": "\\hbar", "ℓ": "\\ell", "∠": "\\angle", "∁": "\\complement",
"℘": "\\wp", "ℵ": "\\aleph", "℧": "\\mho",
# Geometric symbols
"△": "\\triangle", "▲": "\\blacktriangle", "▽": "\\triangledown",
"◁": "\\triangleleft", "▷": "\\triangleright", "◀": "\\blacktriangleleft",
"▶": "\\blacktriangleright",
# Square symbols (comprehensive coverage)
"□": "\\square", # U+25A1 WHITE SQUARE
"■": "\\blacksquare", # U+25A0 BLACK SQUARE
"▪": "\\blacksquare", # U+25AA BLACK SMALL SQUARE
"▫": "\\square", # U+25AB WHITE SMALL SQUARE
"◻": "\\square", # U+25FB WHITE MEDIUM SQUARE
"◼": "\\blacksquare", # U+25FC BLACK MEDIUM SQUARE
"◽": "\\square", # U+25FD WHITE MEDIUM SMALL SQUARE
"◾": "\\blacksquare", # U+25FE BLACK MEDIUM SMALL SQUARE
# Other geometric shapes
# NOTE: ▱ (parallelogram) is in UNICODE_PASSTHROUGH
"◇": "\\diamond", "◆": "\\blacklozenge", "○": "\\circ", "●": "\\bullet",
"☆": "\\star", "★": "\\bigstar", "◯": "\\bigcirc",
# Temperature symbols
"℃": "^\\circ\\mathrm{C}", "℉": "^\\circ\\mathrm{F}",
# Basic operators and punctuation
"∶": ":", # U+2236 RATIO to colon for assignment operators
"§": "\\S", "†": "\\dagger", "‵": "\\backprime",
"`": "\\backprime", # U+0060 GRAVE ACCENT (backtick) used as backprime in equations
"―": "\\textemdash", # U+2015 horizontal bar
# Multiplication and division
"×": "\\times", "⋅": "\\cdot", "·": "\\cdot", "÷": "\\div", "∕": "/", "⁄": "/",
"//": "/ /", # Double slash with spacing
"⧸": "/", # U+29F8 BIG SOLIDUS (used in \not patterns but also standalone)
# Mathematical script letters
"ℑ": "\\Im", # Imaginary part (Fraktur I)
"ℜ": "\\Re", # Real part (Fraktur R)
# Arrows (basic)
"→": "\\rightarrow", "←": "\\leftarrow", "↔": "\\leftrightarrow",
"⇒": "\\Longrightarrow", "⇐": "\\Longleftarrow", "⇔": "\\Longleftrightarrow",
"⟹": "\\Longrightarrow", "⟸": "\\Longleftarrow", "⟺": "\\Longleftrightarrow",
"⟷": "\\longleftrightarrow",
"↑": "\\uparrow", "↓": "\\downarrow", "⇑": "\\Uparrow", "⇓": "\\Downarrow",
"↕": "\\updownarrow", "⇕": "\\Updownarrow",
# Arrows (diagonal)
"↗": "\\nearrow", "↘": "\\searrow", "↙": "\\swarrow", "↖": "\\nwarrow",
"⤴": "\\nwarrow", "⤵": "\\searrow", "⤡": "\\swarrow", "⤢": "\\nearrow",
# Arrows (special forms)
# NOTE: Some special arrows (↤ mapsfrom, ⟻ longmapsfrom, ⇜ leftsquigarrow, ⇝ rightsquigarrow) are in UNICODE_PASSTHROUGH
"↦": "\\mapsto",
"⟼": "\\longmapsto",
"↪": "\\hookrightarrow", "↩": "\\hookleftarrow",
"⤷": "\\hookrightarrow", "⤶": "\\hookleftarrow",
"↬": "\\looparrowright", "↫": "\\looparrowleft",
"⇀": "\\rightharpoonup", "↼": "\\leftharpoonup",
"↢": "\\leftarrowtail", "↣": "\\rightarrowtail",
"↰": "\\Lsh", "↱": "\\Rsh",
# Arrows (negated)
"↛": "\\nrightarrow", "↚": "\\nleftarrow", "⇏": "\\nRightarrow", "⇍": "\\nLeftarrow",
"↮": "\\nleftrightarrow", "⇎": "\\nLeftrightarrow",
# Set theory and logic
# Set membership
"∈": "\\in", "∉": "\\notin", "∋": "\\ni", "∌": "\\notni",
"⋲": "\\in", "⋳": "\\ni", "⋴": "\\in", "⋵": "\\ni",
"⋶": "\\in", "⋷": "\\ni", "⋸": "\\pitchfork", "⋹": "\\ltimes",
# Subset and superset relations (basic)
"⊂": "\\subset", "⊃": "\\supset",
"⊆": "\\subseteq", "⊇": "\\supseteq",
"⊄": "\\not\\subset", "⊅": "\\not\\supset",
"⊈": "\\nsubseteq", "⊉": "\\nsupseteq",
"⊊": "\\subsetneq", "⊋": "\\supsetneq",
# Subset and superset relations (extended with double lines)
"⫅": "\\subseteqq", "⫆": "\\supseteqq", # U+2AC5, U+2AC6 subset/supset with double equals
"⫋": "\\subsetneqq", "⫌": "\\supsetneqq", # U+2ACB, U+2ACC not equal variant with double
"⫏": "\\sqsubset", "⫐": "\\sqsupset", # U+2ACF, U+2AD0 square subset/supset
"⫑": "\\sqsubseteq", "⫒": "\\sqsupseteq", # U+2AD1, U+2AD2 square subset/supset equal
"⊏": "\\sqsubset", "⊐": "\\sqsupset", # U+228F, U+2290 square variants
"⊑": "\\sqsubseteq", "⊒": "\\sqsupseteq", # U+2291, U+2292 square equal variants
# Set operations
# NOTE: Some symbols like ⊌ (dotcup), ⊍ (cupdot) are in UNICODE_PASSTHROUGH
"∪": "\\cup", "∩": "\\cap", "∖": "\\setminus",
"⊎": "\\uplus", "⊔": "\\sqcup", "⊓": "\\sqcap",
"⨃": "\\Cup", "⨄": "\\biguplus", "⨅": "\\bigsqcap", "⨆": "\\bigsqcup",
"⩁": "\\Cup", "⩂": "\\Cap", "⩃": "\\cupdot",
# Logical operators
"∧": "\\wedge", "∨": "\\vee", "¬": "\\neg", "⊕": "\\oplus",
"⊗": "\\otimes", "⊙": "\\odot", "⊖": "\\ominus", "⊘": "\\oslash",
"∀": "\\forall", "∃": "\\exists", "∄": "\\nexists",
"⊤": "\\top", "⊥": "\\perp", "⊢": "\\vdash", "⊣": "\\dashv",
"⊨": "\\models", "⊭": "\\not\\models",
"∴": "\\therefore", "∵": "\\because",
"⋏": "\\curlywedge", "⋎": "\\curlyvee",
# Additional logical symbols
"⊩": "\\Vdash", "⊪": "\\Vvdash", "⊫": "\\VDash", "⊬": "\\nvdash",
"⊯": "\\nVdash", "⊮": "\\nVDash", "∎": "\\qed", "◊": "\\lozenge",
"≐": "\\doteq", "≑": "\\doteqdot", "≒": "\\fallingdotseq", "≓": "\\risingdotseq",
# Relations
# Equality and equivalence
"≈": "\\approx", "≅": "\\cong", "≡": "\\equiv", "≢": "\\not\\equiv",
"≠": "\\neq", "≁": "\\not\\sim", "≄": "\\nsim", "≇": "\\ncong",
"~": "\\sim", # ASCII tilde (U+007E) - similarity operator
"∼": "\\sim", "≃": "\\simeq", "≂": "\\eqsim", "≊": "\\approxeq",
# Inequality relations (basic)
# NOTE: Triple relations ⋘ (lll), ⋙ (ggg) are in UNICODE_PASSTHROUGH
"≤": "\\leqslant", "≥": "\\geqslant", "≮": "\\nless", "≯": "\\ngtr",
"≰": "\\nleq", "≱": "\\ngeq", "≪": "\\ll", "≫": "\\gg",
# Inequality relations (extended)
"≦": "\\leqq", "≧": "\\geqq", "≨": "\\lneqq", "≩": "\\gneqq",
"⩽": "\\leqslant", "⩾": "\\geqslant", "⩿": "\\gneq",
"≶": "\\lessgtr", "≷": "\\gtrless", "≸": "\\not\\lessgtr", "≹": "\\not\\gtrless",
# NOTE: Advanced comparison symbols (⪋ lesseqgtr, ⪌ gtreqless, ⪅ lessapprox, etc.) are in UNICODE_PASSTHROUGH
"⪇": "\\lneq", "⪈": "\\gneq",
# Precedence and succession
# NOTE: Advanced precedence symbols (⋞ curlyeqprec, ⋟ curlyeqsucc, ⪷ precapprox, ⪸ succapprox) are in UNICODE_PASSTHROUGH
"≺": "\\prec", "≻": "\\succ", "⪯": "\\preceq", "⪰": "\\succeq",
"⪳": "\\preceqq", "⪴": "\\succeqq",
"⪵": "\\precneqq", "⪶": "\\succneqq",
# Similarity and comparison
"≲": "\\lesssim", "≳": "\\gtrsim", "≴": "\\not\\lesssim", "≵": "\\not\\gtrsim",
"∝": "\\propto", "∥": "\\parallel", "∦": "\\nparallel",
"≍": "\\asymp", "≎": "\\Bumpeq", "≏": "\\bumpeq",
"∽": "\\backsim", "⋍": "\\backsimeq", # U+22CD REVERSED TILDE EQUALS
"≌": "\\cong", # U+224C ALL EQUAL TO (commonly used for congruence)
"⊲": "\\lhd", "⊳": "\\rhd",
"⌢": "\\frown", "⌣": "\\smile",
# Special equality symbols
# NOTE: Some special equality symbols (≔ coloneqq, ≕ eqqcolon, ≘ arceq, ≙ wedgeq, ≚ veeeq) are in UNICODE_PASSTHROUGH
"≜": "\\triangleq", "≝": "\\overset{\\mathrm{def}}{=}", "≞": "\\overset{\\mathrm{m}}{=}",
"≟": "\\overset{?}{=}", "≗": "\\circeq", "≖": "\\eqcirc",
# Calculus and analysis
# Integrals
# NOTE: Many advanced integral symbols (∯, ∰, ∱, etc.) are in UNICODE_PASSTHROUGH
# for direct MathJax rendering, so they are NOT listed here
"∫": "\\int", "∬": "\\iint", "∭": "\\iiint", "⨌": "\\iiiint",
"∮": "\\oint",
# Differentials and operators
"∆": "\\Delta", "∂": "\\partial", "∇": "\\nabla",
"∝": "\\propto", "∞": "\\infty",
# Cardinality symbols
# NOTE: Hebrew letters ℶ (beth), ℷ (gimel), ℸ (daleth) are in UNICODE_PASSTHROUGH
"ℵ": "\\aleph",
# Number sets (use \mathbb for blackboard bold notation)
"ℕ": "\\mathbb{N}", "ℤ": "\\mathbb{Z}", "ℚ": "\\mathbb{Q}",
"ℝ": "\\mathbb{R}", "ℂ": "\\mathbb{C}", "ℍ": "\\mathbb{H}",
"ℙ": "\\mathbb{P}", "𝔽": "\\mathbb{F}", "𝔾": "\\mathbb{G}",
# Special operators
# Basic arithmetic operators
"±": "\\pm", "∓": "\\mp", "−": "-", "∔": "\\dotplus",
# Binary operators (algebraic)
"⋆": "\\star", "∗": "\\ast", "∘": "\\circ", "∙": "\\bullet",
"⋄": "\\diamond", "⋅": "\\cdot", "•": "\\bullet",
# Binary operators (lattice and semilattice)
# NOTE: ⊌ (dotcup), ⊍ (cupdot) are in UNICODE_PASSTHROUGH
"⊔": "\\sqcup", "⊓": "\\sqcap", "⊎": "\\uplus",
# Binary operators (group theory)
# NOTE: Some group theory operators (⋋ leftthreetimes, ⋌ rightthreetimes, ⋇ divideontimes) are in UNICODE_PASSTHROUGH
"⋉": "\\ltimes", "⋊": "\\rtimes",
"⨿": "\\amalg", "⋈": "\\bowtie",
# Binary operators (wreath and semidirect products)
"≀": "\\wr", "⊺": "\\intercal", "⊻": "\\veebar",
# Binary operators (extended)
# NOTE: Advanced box operators (⧄ boxbox, ⧅ boxast, ⧆ boxcirc, ⧇ boxbslash, ⧈ boxslash) are in UNICODE_PASSTHROUGH
"⊝": "\\ominus", "⊞": "\\boxplus", "⊟": "\\boxminus", "⊠": "\\boxtimes",
"⊡": "\\boxdot",
# Big operators
"∑": "\\sum", "∏": "\\prod", "∐": "\\coprod", "⋃": "\\bigcup",
"⋂": "\\bigcap", "⋁": "\\bigvee", "⋀": "\\bigwedge", "⨁": "\\bigoplus",
"⨂": "\\bigotimes", "⨄": "\\biguplus", "⨅": "\\bigsqcap", "⨆": "\\bigsqcup",
# Delimiters and brackets
# Angle brackets
"⟨": "\\langle", "⟩": "\\rangle", "⟪": "\\langle\\langle", "⟫": "\\rangle\\rangle",
"〈": "\\langle", "〉": "\\rangle", # CJK angle brackets
# Floor and ceiling
"⌊": "\\lfloor", "⌋": "\\rfloor", "⌈": "\\lceil", "⌉": "\\rceil",
# Special brackets
# NOTE: Some special brackets (⦅ llparenthesis, ⦆ rrparenthesis, 〚 llbracket, 〛 rrbracket, ⦉ Langle, ⦊ Rangle) are in UNICODE_PASSTHROUGH
"⦃": "\\{", "⦄": "\\}",
"⟦": "[\\![", "⟧": "]\\!]", "⟬": "\\llbracket", "⟭": "\\rrbracket",
"⦇": "\\llparenthesis", "⦈": "\\rrparenthesis",
# Absolute value and norm
"|": "|", "‖": "\\|", "∥": "\\parallel", # ‖ (U+2016) is double vertical line for norms → \|
# Miscellaneous symbols
# NOTE: Radical symbols must include {} braces for valid LaTeX
# Users may type these directly as text (malformed input), so we ensure valid output
"√": "\\sqrt{}", "∛": "\\sqrt[3]{}", "∜": "\\sqrt[4]{}",
"∡": "\\measuredangle", "∢": "\\sphericalangle", "°": "^\\circ",
# Prime notation - both Unicode primes and ASCII apostrophe (commonly used in Word)
"'": "\\prime", # ASCII apostrophe (U+0027) - commonly typed instead of Unicode prime
"′": "\\prime", "″": "\\prime\\prime", "‴": "\\prime\\prime\\prime", "⁗": "\\prime\\prime\\prime\\prime",
"℮": "e", "ℯ": "e", "ⅈ": "i", "ⅉ": "j",
# NOTE: ⋰ (iddots) is in UNICODE_PASSTHROUGH
"…": "\\ldots", "⋯": "\\cdots", "⋮": "\\vdots", "⋱": "\\ddots",
"♭": "\\flat", "♮": "\\natural", "♯": "\\sharp",
# Fullwidth punctuation (CJK)
"，": ", ", # U+FF0C fullwidth comma (Chinese comma) → regular comma with space
# Accent marks (for detection in mover elements)
"¨": "__DDOT__", # U+00A8 diaeresis (standalone) - placeholder for ddot detection
"⃛": "__DDDOT__", # U+20DB combining three dots above - placeholder for dddot detection
"̈": "__DDOT__", # U+0308 combining diaeresis - placeholder for ddot detection
"⃛": "__DDDOT__", # U+20DB combining triple dot above - placeholder for dddot detection
# Chemistry arrows and symbols (harpoons)
"⇌": "\\rightleftharpoons", # U+21CC rightleftharpoons (equilibrium)
"⇋": "\\leftrightharpoons", # U+21CB leftrightharpoons (equilibrium reversed)
"⇀": "\\rightharpoonup", # U+21C0 rightharpoonup
"⇁": "\\rightharpoondown", # U+21C1 rightharpoondown
"↼": "\\leftharpoonup", # U+21BC leftharpoonup
"↽": "\\leftharpoondown", # U+21BD leftharpoondown
"⥂": "\\rightleftharpoons", # U+2942 alternate form
"⥃": "\\leftrightharpoons", # U+2943 alternate form
"⥄": "\\rightleftarrows", # U+2944 right over left arrows
"⥅": "\\leftrightarrows", # U+2945 left over right arrows
"⥆": "\\rightleftarrows", # U+2946 right over left arrows
"⥇": "\\leftrightarrows", # U+2947 left over right arrows
"⇄": "\\rightleftarrows", # U+21C4 right over left (already in extended arrows)
"⇆": "\\leftrightarrows", # U+21C6 left over right (already in extended arrows)
"⇉": "\\rightrightarrows", # U+21C9 double right (already in extended arrows)
"⇇": "\\leftleftarrows", # U+21C7 double left (already in extended arrows)
# Overbrace and underbrace characters
"⏞": "\\overbrace", "⏟": "\\underbrace",
# Extended mathematical operators (remaining unique ones)
# NOTE: ⊚ (ocirc), ⊛ (oast) are in UNICODE_PASSTHROUGH
# Extended relation symbols (remaining unique ones)
# NOTE: Several special equality symbols (≔ coloneqq, ≕ eqqcolon, ≘ arceq, ≙ wedgeq, ≚ veeeq) are in UNICODE_PASSTHROUGH
"≆": "\\approxeq", "≛": "\\triangleq",
# Extended arrows (remaining unique ones)
"⇈": "\\upuparrows", "⇊": "\\downdownarrows",
"⇅": "\\updownarrows", "⇵": "\\downuparrows", "↶": "\\curvearrowleft",
"↷": "\\curvearrowright", "↺": "\\circlearrowleft", "↻": "\\circlearrowright",
# Extended logical symbols
# NOTE: ⟀ (lozengeminus) is in UNICODE_PASSTHROUGH
"⟁": "\\lozengepls", "⟂": "\\perp", "⟃": "\\between",
"⟄": "\\pitchfork", "⟅": "\\varpropto", "⟆": "\\backprime", "⟇": "\\caretinsert",
# Extended geometric symbols
"⟐": "\\angle", "⟑": "\\measuredangle", "⟒": "\\sphericalangle", "⟓": "\\rightangle",
"⟔": "\\angle", "⟕": "\\measuredangle", "⟖": "\\sphericalangle", "⟗": "\\rightangle",
# Extended miscellaneous symbols
"⟘": "\\perp", "⟙": "\\diamondsuit", "⟚": "\\heartsuit", "⟛": "\\clubsuit",
"⟜": "\\spadesuit", "⟝": "\\varnothing", "⟞": "\\diameter", "⟟": "\\house",
}
# Additional Symbol Replacements (Consolidated from scattered code)
# ========================================================================
# These symbols are used in specific contexts (e.g., function subscripts)
# and should be applied consistently across the codebase
ADDITIONAL_SYMBOL_REPLACEMENTS = {
# Arrows used in function subscripts (line 1828)
'→': '\\to ',
# These are handled in UNICODE_TO_LATEX but kept for reference in subscript context
# '∞': '\\infty', # Already in UNICODE_TO_LATEX
# '≤': '\\leqslant ', # Already in UNICODE_TO_LATEX
# '≥': '\\geqslant ', # Already in UNICODE_TO_LATEX
# '≠': '\\ne ', # Already in UNICODE_TO_LATEX as \\neq
# '±': '\\pm ', # Already in UNICODE_TO_LATEX
}
# LaTeX command normalization (converting between command variants)
LATEX_COMMAND_REPLACEMENTS = {
# Logical equivalence (line 5545)
'\\Longleftrightarrow': '\\iff ',
# Arrow shorthand (line 5548)
'\\lra': '\\leftrightarrow',
}
# Summation symbols for OMML fallback parser
SUMMATION_SYMBOLS = {
'∑': 'sum',
'∏': 'prod',
'∫': 'integral',
'Σ': 'sum',
'Π': 'prod'
}
# LaTeX commands not supported by MathJax - use Unicode fallback
# Maps LaTeX command (without backslash) to Unicode character
# This comprehensive list ensures proper rendering in MathJax environments
MATHJAX_UNSUPPORTED_TO_UNICODE = {
# =========================================================================
# INTEGRAL OPERATORS - Many not in standard MathJax
# =========================================================================
# Surface and volume integrals
'oiint': '∯', # U+222F SURFACE INTEGRAL
'oiiint': '∰', # U+2230 VOLUME INTEGRAL
# Clockwise/counterclockwise integrals - NOT in MathJax
'intclockwise': '∱', # U+2231 CLOCKWISE INTEGRAL
'varointclockwise': '∲', # U+2232 CLOCKWISE CONTOUR INTEGRAL
'varointctrclockwise': '∳', # U+2233 ANTICLOCKWISE CONTOUR INTEGRAL
# Extended integrals - limited MathJax support
'fint': '⨍', # U+2A0D FINITE PART INTEGRAL
'pointint': '⨎', # U+2A0E PATH INTEGRAL (line integral)
'sqint': '⨏', # U+2A0F INTEGRAL WITH SQUARED OPERATOR
'intlarhk': '⨐', # U+2A10 CIRCULATION FUNCTION
'intbar': '⨑', # U+2A11 INTEGRAL WITH OVERBAR
'intBar': '⨒', # U+2A12 INTEGRAL WITH DOUBLE OVERBAR
'fintbar': '⨓', # U+2A13 FINITE PART INTEGRAL WITH OVERBAR
'ointbar': '⨔', # U+2A14 CONTOUR INTEGRAL WITH OVERBAR
'ointBar': '⨕', # U+2A15 CONTOUR INTEGRAL WITH DOUBLE OVERBAR
'oiintbar': '⨖', # U+2A16 SURFACE INTEGRAL WITH OVERBAR
'oiintBar': '⨗', # U+2A17 SURFACE INTEGRAL WITH DOUBLE OVERBAR
# =========================================================================
# BINARY OPERATORS - Non-standard or limited support
# =========================================================================
# Division operator
'divideontimes': '⋇', # U+22C7 DIVISION TIMES
# Circle operators - non-standard names
'oast': '⊛', # U+229B CIRCLED ASTERISK OPERATOR
'ocirc': '⊚', # U+229A CIRCLED RING OPERATOR
# Box operators - extended (standard ones like boxplus ARE supported)
'boxbox': '⧄', # U+29C4 SQUARED SQUARE
'boxast': '⧅', # U+29C5 SQUARED ASTERISK
'boxcirc': '⧆', # U+29C6 SQUARED SMALL CIRCLE
'boxbslash': '⧇', # U+29C7 SQUARED BACKSLASH
'boxslash': '⧈', # U+29C8 SQUARED SLASH
# Semidirect products
'leftthreetimes': '⋋', # U+22CB LEFT SEMIDIRECT PRODUCT
'rightthreetimes': '⋌', # U+22CC RIGHT SEMIDIRECT PRODUCT
# Set operations with dot
'dotcup': '⊌', # U+228C MULTISET
'cupdot': '⊍', # U+228D MULTISET MULTIPLICATION
# =========================================================================
# ARROWS - Extended arrows with limited MathJax support
# =========================================================================
'mapsfrom': '↤', # U+21A4 LEFTWARDS ARROW FROM BAR
'longmapsfrom': '⟻', # U+27FB LONG LEFTWARDS ARROW FROM BAR
'leftsquigarrow': '⇜', # U+21DC LEFTWARDS SQUIGGLE ARROW
'rightsquigarrow': '⇝', # U+21DD RIGHTWARDS SQUIGGLE ARROW
# 'looparrowleft': '↫', # U+21AB LEFTWARDS ARROW WITH LOOP
# 'looparrowright': '↬', # U+21AC RIGHTWARDS ARROW WITH LOOP
# 'leftarrowtail': '↢', # U+21A2 LEFTWARDS ARROW WITH TAIL
# 'rightarrowtail': '↣', # U+21A3 RIGHTWARDS ARROW WITH TAIL
# 'Lsh': '↰', # U+21B0 UPWARDS ARROW WITH TIP LEFTWARDS
# 'Rsh': '↱', # U+21B1 UPWARDS ARROW WITH TIP RIGHTWARDS
# 'curvearrowleft': '↶', # U+21B6 ANTICLOCKWISE TOP SEMICIRCLE ARROW
# 'curvearrowright': '↷', # U+21B7 CLOCKWISE TOP SEMICIRCLE ARROW
# 'circlearrowleft': '↺', # U+21BA ANTICLOCKWISE OPEN CIRCLE ARROW
# 'circlearrowright': '↻', # U+21BB CLOCKWISE OPEN CIRCLE ARROW
# 'upuparrows': '⇈', # U+21C8 UPWARDS PAIRED ARROWS
# 'downdownarrows': '⇊', # U+21CA DOWNWARDS PAIRED ARROWS
# 'updownarrows': '⇅', # U+21C5 UPWARDS ARROW LEFTWARDS OF DOWNWARDS
# 'downuparrows': '⇵', # U+21F5 DOWNWARDS ARROW LEFTWARDS OF UPWARDS
# =========================================================================
# RELATION SYMBOLS - Extended relations
# =========================================================================
# Triple comparisons
'lll': '⋘', # U+22D8 VERY MUCH LESS-THAN
'ggg': '⋙', # U+22D9 VERY MUCH GREATER-THAN
# Combined relations
'lesseqgtr': '⪋', # U+2A8B LESS-THAN EQUAL TO OR GREATER-THAN
'gtreqless': '⪌', # U+2A8C GREATER-THAN EQUAL TO OR LESS-THAN
'lessapprox': '⪅', # U+2A85 LESS-THAN OR APPROXIMATE
'gtrapprox': '⪆', # U+2A86 GREATER-THAN OR APPROXIMATE
'lnapprox': '⪉', # U+2A89 LESS-THAN AND NOT APPROXIMATE
'gnapprox': '⪊', # U+2A8A GREATER-THAN AND NOT APPROXIMATE
# Precedence relations
'precapprox': '⪷', # U+2AB7 PRECEDES ABOVE ALMOST EQUAL TO
'succapprox': '⪸', # U+2AB8 SUCCEEDS ABOVE ALMOST EQUAL TO
'curlyeqprec': '⋞', # U+22DE EQUAL TO OR PRECEDES
'curlyeqsucc': '⋟', # U+22DF EQUAL TO OR SUCCEEDS
# Similarity relations
# 'backsim': '∽', # U+223D REVERSED TILDE
# 'backsimeq': '⋍', # U+22CD REVERSED TILDE EQUALS
# Special equality symbols
'wedgeq': '≙', # U+2259 ESTIMATES (wedge equals)
'veeeq': '≚', # U+225A EQUIANGULAR TO
'arceq': '≘', # U+2258 CORRESPONDS TO
'coloneqq': '≔', # U+2254 COLON EQUALS
'eqqcolon': '≕', # U+2255 EQUALS COLON
'Coloneqq': '⩴', # U+2A74 DOUBLE COLON EQUAL
'eqcolon': '∹', # U+2239 EXCESS (-:)
# =========================================================================
# BRACKET/DELIMITER SYMBOLS
# =========================================================================
'llbracket': '〚', # U+301A LEFT WHITE SQUARE BRACKET
'rrbracket': '〛', # U+301B RIGHT WHITE SQUARE BRACKET
'llparenthesis': '⦅', # U+2985 LEFT WHITE PARENTHESIS
'rrparenthesis': '⦆', # U+2986 RIGHT WHITE PARENTHESIS
'Langle': '⦉', # U+2989 Z NOTATION LEFT BINDING BRACKET
'Rangle': '⦊', # U+298A Z NOTATION RIGHT BINDING BRACKET
# =========================================================================
# GEOMETRIC AND MISCELLANEOUS SYMBOLS
# =========================================================================
'parallelogram': '▱', # U+25B1 WHITE PARALLELOGRAM
'iddots': '⋰', # U+22F0 UP RIGHT DIAGONAL ELLIPSIS
# 'mho': '℧', # U+2127 INVERTED OHM SIGN
# 'complement': '∁', # U+2201 COMPLEMENT
# Hebrew letters (beyond aleph)
'beth': 'ℶ', # U+2136 BET SYMBOL
'gimel': 'ℷ', # U+2137 GIMEL SYMBOL
'daleth': 'ℸ', # U+2138 DALET SYMBOL
# =========================================================================
# LOGICAL AND SET OPERATORS
# =========================================================================
# Extended subset/superset relations
# 'subseteqq': '⫅', # U+2AC5 SUBSET OF ABOVE EQUALS SIGN
# 'supseteqq': '⫆', # U+2AC6 SUPERSET OF ABOVE EQUALS SIGN
# 'subsetneqq': '⫋', # U+2ACB SUBSET OF ABOVE NOT EQUAL TO
# 'supsetneqq': '⫌', # U+2ACC SUPERSET OF ABOVE NOT EQUAL TO
# Logic symbols
'lozengeminus': '⟀', # U+27C0 THREE DIMENSIONAL ANGLE
'between': '≬', # U+226C BETWEEN
}
# Regional symbol preferences (e.g., Chinese region prefers varnothing over emptyset)
# Maps standard LaTeX command to regional alternative
REGIONAL_SYMBOL_PREFERENCES = {
# Chinese region: prefer varnothing (⌀) over emptyset (∅)
# varnothing has a diagonal line which is clearer in Chinese typography
'emptyset': 'varnothing', # ∅ → ⌀
}
# Unicode symbols that should be kept as-is (NOT converted to LaTeX)
# These are symbols whose LaTeX commands are not supported or render poorly in MathJax.
# By keeping them as Unicode, they render correctly using system fonts.
# This set is the "inverse" of MATHJAX_UNSUPPORTED_TO_UNICODE - same symbols, different direction.
UNICODE_PASSTHROUGH = {
# =========================================================================
# INTEGRAL OPERATORS - Keep as Unicode for MathJax compatibility
# =========================================================================
'∯', # U+222F SURFACE INTEGRAL (oiint)
'∰', # U+2230 VOLUME INTEGRAL (oiiint)
'∱', # U+2231 CLOCKWISE INTEGRAL (intclockwise)
'∲', # U+2232 CLOCKWISE CONTOUR INTEGRAL (varointclockwise)
'∳', # U+2233 ANTICLOCKWISE CONTOUR INTEGRAL (varointctrclockwise)
'⨍', # U+2A0D FINITE PART INTEGRAL (fint)
'⨎', # U+2A0E PATH INTEGRAL (pointint)
'⨏', # U+2A0F INTEGRAL WITH SQUARED OPERATOR (sqint)
'⨐', # U+2A10 CIRCULATION FUNCTION (intlarhk)
'⨑', # U+2A11 INTEGRAL WITH OVERBAR (intbar)
'⨒', # U+2A12 INTEGRAL WITH DOUBLE OVERBAR (intBar)
'⨓', # U+2A13 FINITE PART INTEGRAL WITH OVERBAR (fintbar)
'⨔', # U+2A14 CONTOUR INTEGRAL WITH OVERBAR (ointbar)
'⨕', # U+2A15 CONTOUR INTEGRAL WITH DOUBLE OVERBAR (ointBar)
'⨖', # U+2A16 SURFACE INTEGRAL WITH OVERBAR (oiintbar)
'⨗', # U+2A17 SURFACE INTEGRAL WITH DOUBLE OVERBAR (oiintBar)
# =========================================================================
# BINARY OPERATORS - Keep as Unicode
# =========================================================================
'⋇', # U+22C7 DIVISION TIMES (divideontimes)
'⊛', # U+229B CIRCLED ASTERISK OPERATOR (oast)
'⊚', # U+229A CIRCLED RING OPERATOR (ocirc)
'⧄', # U+29C4 SQUARED SQUARE (boxbox)
'⧅', # U+29C5 SQUARED ASTERISK (boxast)
'⧆', # U+29C6 SQUARED SMALL CIRCLE (boxcirc)
'⧇', # U+29C7 SQUARED BACKSLASH (boxbslash)
'⧈', # U+29C8 SQUARED SLASH (boxslash)
'⋋', # U+22CB LEFT SEMIDIRECT PRODUCT (leftthreetimes)
'⋌', # U+22CC RIGHT SEMIDIRECT PRODUCT (rightthreetimes)
'⊌', # U+228C MULTISET (dotcup)
'⊍', # U+228D MULTISET MULTIPLICATION (cupdot)
# =========================================================================
# ARROWS - Keep as Unicode for consistency
# =========================================================================
'↤', # U+21A4 LEFTWARDS ARROW FROM BAR (mapsfrom)
'⟻', # U+27FB LONG LEFTWARDS ARROW FROM BAR (longmapsfrom)
'⇜', # U+21DC LEFTWARDS SQUIGGLE ARROW (leftsquigarrow)
'⇝', # U+21DD RIGHTWARDS SQUIGGLE ARROW (rightsquigarrow)
# '↫', # U+21AB LEFTWARDS ARROW WITH LOOP (looparrowleft)
# '↬', # U+21AC RIGHTWARDS ARROW WITH LOOP (looparrowright)
# '↢', # U+21A2 LEFTWARDS ARROW WITH TAIL (leftarrowtail)
# '↣', # U+21A3 RIGHTWARDS ARROW WITH TAIL (rightarrowtail)
# '↰', # U+21B0 UPWARDS ARROW WITH TIP LEFTWARDS (Lsh)
# '↱', # U+21B1 UPWARDS ARROW WITH TIP RIGHTWARDS (Rsh)
# '↶', # U+21B6 ANTICLOCKWISE TOP SEMICIRCLE ARROW (curvearrowleft)
# '↷', # U+21B7 CLOCKWISE TOP SEMICIRCLE ARROW (curvearrowright)
# '↺', # U+21BA ANTICLOCKWISE OPEN CIRCLE ARROW (circlearrowleft)
# '↻', # U+21BB CLOCKWISE OPEN CIRCLE ARROW (circlearrowright)
# '⇈', # U+21C8 UPWARDS PAIRED ARROWS (upuparrows)
# '⇊', # U+21CA DOWNWARDS PAIRED ARROWS (downdownarrows)
# '⇅', # U+21C5 UPWARDS ARROW LEFTWARDS OF DOWNWARDS (updownarrows)
# '⇵', # U+21F5 DOWNWARDS ARROW LEFTWARDS OF UPWARDS (downuparrows)
# =========================================================================
# RELATION SYMBOLS - Keep as Unicode
# =========================================================================
'⋘', # U+22D8 VERY MUCH LESS-THAN (lll)
'⋙', # U+22D9 VERY MUCH GREATER-THAN (ggg)
'⪋', # U+2A8B LESS-THAN EQUAL TO OR GREATER-THAN (lesseqgtr)
'⪌', # U+2A8C GREATER-THAN EQUAL TO OR LESS-THAN (gtreqless)
'⪅', # U+2A85 LESS-THAN OR APPROXIMATE (lessapprox)
'⪆', # U+2A86 GREATER-THAN OR APPROXIMATE (gtrapprox)
'⪉', # U+2A89 LESS-THAN AND NOT APPROXIMATE (lnapprox)
'⪊', # U+2A8A GREATER-THAN AND NOT APPROXIMATE (gnapprox)
'⪷', # U+2AB7 PRECEDES ABOVE ALMOST EQUAL TO (precapprox)
'⪸', # U+2AB8 SUCCEEDS ABOVE ALMOST EQUAL TO (succapprox)
'⋞', # U+22DE EQUAL TO OR PRECEDES (curlyeqprec)
'⋟', # U+22DF EQUAL TO OR SUCCEEDS (curlyeqsucc)
# '∽', # U+223D REVERSED TILDE (backsim)
# '⋍', # U+22CD REVERSED TILDE EQUALS (backsimeq)
'≙', # U+2259 ESTIMATES (wedgeq)
'≚', # U+225A EQUIANGULAR TO (veeeq)
'≘', # U+2258 CORRESPONDS TO (arceq)
'≔', # U+2254 COLON EQUALS (coloneqq)
'≕', # U+2255 EQUALS COLON (eqqcolon)
'⩴', # U+2A74 DOUBLE COLON EQUAL (Coloneqq)
'∹', # U+2239 EXCESS (eqcolon)
'≬', # U+226C BETWEEN (between)
# =========================================================================
# BRACKET/DELIMITER SYMBOLS - Keep as Unicode
# =========================================================================
'〚', # U+301A LEFT WHITE SQUARE BRACKET (llbracket)
'〛', # U+301B RIGHT WHITE SQUARE BRACKET (rrbracket)
'⦅', # U+2985 LEFT WHITE PARENTHESIS (llparenthesis)
'⦆', # U+2986 RIGHT WHITE PARENTHESIS (rrparenthesis)
'⦉', # U+2989 Z NOTATION LEFT BINDING BRACKET (Langle)
'⦊', # U+298A Z NOTATION RIGHT BINDING BRACKET (Rangle)
# =========================================================================
# GEOMETRIC AND MISCELLANEOUS SYMBOLS - Keep as Unicode
# =========================================================================
'▱', # U+25B1 WHITE PARALLELOGRAM (parallelogram)
'⋰', # U+22F0 UP RIGHT DIAGONAL ELLIPSIS (iddots)
# '℧', # U+2127 INVERTED OHM SIGN (mho)
# '∁', # U+2201 COMPLEMENT (complement)
'ℶ', # U+2136 BET SYMBOL (beth)
'ℷ', # U+2137 GIMEL SYMBOL (gimel)
'ℸ', # U+2138 DALET SYMBOL (daleth)
'⟀', # U+27C0 THREE DIMENSIONAL ANGLE (lozengeminus)
# =========================================================================
# LOGICAL AND SET OPERATORS - Keep as Unicode
# =========================================================================
# '⫅', # U+2AC5 SUBSET OF ABOVE EQUALS SIGN (subseteqq)
# '⫆', # U+2AC6 SUPERSET OF ABOVE EQUALS SIGN (supseteqq)
# '⫋', # U+2ACB SUBSET OF ABOVE NOT EQUAL TO (subsetneqq)
# '⫌', # U+2ACC SUPERSET OF ABOVE NOT EQUAL TO (supsetneqq)
}

63
scripts/omml_mathml2latex/data/units.py Normal file
View File

@@ -0,0 +1,63 @@
"""
Common unit definitions for chemistry, physics, and mathematics.
Contains:
- COMMON_UNITS: Set of common units that should be formatted as \\mathrm{unit}
Includes: chemistry units, SI base units, SI derived units, non-SI units,
angular units, time units, concentration units, biological units
"""
# Common units that should be formatted as ~\mathrm{unit}~ (with preceding and trailing space)
COMMON_UNITS = {
# Chemistry units
'mol', 'M', 'mM', 'µM', 'nM', 'pM', 'pH', 'pOH', 'atm', 'bar', 'Torr', 'eq',
'aq', 'sol', 'ppm', 'ppb', 'vol', 'wt',
# SI base units
'kg', 'g', 'mg', 'µg', 'ng', 'pg', 'ton', 'tonne',
'm', 'km', 'cm', 'mm', 'µm', 'nm', 'pm', 'Å',
's', 'ms', 'µs', 'ns', 'ps', 'min', 'h', 'hr', 'day',
'A', 'mA', 'µA', 'nA', 'pA',
'K', 'mK', 'µK', 'nK',
'cd', 'lm', 'lx',
# SI derived units
'Hz', 'kHz', 'MHz', 'GHz', 'THz',
'N', 'kN', 'MN', 'µN', 'nN', 'pN',
'Pa', 'kPa', 'MPa', 'GPa', 'hPa', 'mbar',
'J', 'kJ', 'MJ', 'µJ', 'nJ', 'pJ', 'mJ',
'W', 'kW', 'MW', 'GW', 'mW', 'µW', 'nW', 'pW',
'C', 'mC', 'µC', 'nC', 'pC',
'V', 'kV', 'MV', 'mV', 'µV', 'nV',
'F', 'mF', 'µF', 'nF', 'pF',
'Ω', 'kΩ', 'MΩ', 'GΩ', 'mΩ', 'µΩ',
'S', 'mS', 'µS', 'nS',
'Wb', 'mWb', 'µWb', 'nWb',
'T', 'mT', 'µT', 'nT', 'pT',
'H', 'mH', 'µH', 'nH', 'pH',
# Common non-SI units
'L', 'mL', 'µL', 'nL', 'pL', 'dL', 'cL',
'°C', '°F', '°K', 'celsius', 'fahrenheit', 'kelvin',
'eV', 'keV', 'MeV', 'GeV', 'TeV', 'meV', 'µeV',
'cal', 'kcal', 'Cal',
'Bq', 'kBq', 'MBq', 'GBq', 'mBq',
'Gy', 'mGy', 'µGy', 'kGy',
'Sv', 'mSv', 'µSv', 'kSv',
# Angular units
'rad', 'mrad', 'µrad', 'sr',
'deg', 'arcmin', 'arcsec',
# Time units
'ms', 'µs', 'ns', 'ps', 'fs', 'as',
'year', 'yr', 'month', 'week', 'day', 'hr', 'min',
# Concentration and biological units
'IU', 'U', 'unit', 'units', 'Da', 'kDa', 'MDa',
'bp', 'kbp', 'Mbp', 'kb', 'Mb', 'Gb',
'rpm', 'rps', 'Hz', 'cps',
# Physics constants (when used as units)
'c', 'h', 'ħ', 'kB', 'NA', 'R', 'σ', 'ε₀', 'μ₀',
}