Quentin/DoLuEdu-WordFileParse
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

init: WordFileParse docx→HTML 解析服务（含 Word EQ 域公式修复）

Browse Source 
服务：Spring Boot 3.2 + docx4j 8.3.9，POST /api/v3/server_office_file/file/parse
      接受 .docx，返回 HTML（公式 → LaTeX，图片 → RustFS 外链）

公式支持三类来源：
- OMML（Word 2007+ 公式）      → Python omml_mathml2latex
- MathType OLE 对象            → Ruby mathtype2mathml → MathML → LaTeX
- Word EQ 域（老教辅主力）      → EqToLatexUtil（本次新增）

本次改动：
1. 新增 java-svc/.../core/EqToLatexUtil.java：
   移植自 jby backend-services office-core，EQ 语法 → LaTeX
   （\f→\dfrac、\r→\sqrt、\i→\int/\sum/\prod、\s→^{}/_{}、\b→\left\right、
     \a→array、\x→\boxed/\overline、\o\al 堆叠、圈码、化学符号、极限、连等式）
   品牌标记 JBYU* → WPU*，与采集端一致
2. CtoMathFinder：接上 EQ 分支（原为 not-ported 桩），域代码经
   EqToLatexUtil.convert 转 LaTeX 写入 eqLatexMap（WPEQNUM 占位符）
3. WordParseUtil.dealPlaceholder：EQ 的 LaTeX 同样加 \( \) 包裹
   （原先因 WPEQNUM 例外不加包裹，前端/MathJax 不渲染）

修前症状：教辅 Word 里分式/方程组/上下标整段丢失（如 {−1/3, 1} 变成 {－，1}）
修后实测：46 份教辅 2908 处 EQ 域全部还原，公式总数 2954 处、零未包裹残留

安全：application.yml（含 RustFS 密钥）不入库，提供 application.yml.example；
      OssService 的 @Value 默认密钥已清空
This commit is contained in:
Quentin
2026-09-29 09:43:13 +08:00
commit 3271fee5cc
360 changed files with 86082 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

388
scripts/omml_mathml2latex/visitors/mo_visitor.py Normal file
View File

@@ -0,0 +1,388 @@
"""
Visitor for <mo> (operator) MathML elements.
The <mo> element represents operators in MathML, including:
- Binary operators (+, -, ×, ÷)
- Relational operators (=, <, >, ≤, ≥)
- Delimiters (parentheses, brackets, braces)
- Special operators (∫, ∑, ∏, ∇)
"""
from lxml import etree as ET
from ..core.interfaces import Visitor, Strategy
from ..utils import text_content as _text_content, local_name as _local, convert_spaces_to_latex, escape_latex_special_chars
from ..utils.debug import debug_print
from ..data import CHEMICAL_ELEMENTS, COMMON_UNITS
from ..converters.post_processor import PostProcessor
# Operators with built-in LaTeX spacing - skip manual spaces after these
SKIP_SPACE_AFTER = {
# Relation operators (thick spacing in LaTeX)
'\\rightarrow', '\\leftarrow', '\\to', '→', '←',
'\\Longrightarrow', '\\Longleftarrow', '⇒', '⇐',
'\\leftrightarrow', '\\Longleftrightarrow', '↔', '⇔', '⟷',
'=', '\\equiv', '\\approx', '≈', '\\cong', '≅',
'<', '>', '\\leq', '\\geq', '≤', '≥', '\\neq', '≠',
'\\subseteq', '⊆', '\\supseteq', '⊇',
'\\in', '∈', '\\notin', '∉',
'\\perp', '⊥', '\\parallel', '∥',
'\\sim', '∼', '\\simeq', '≃', '\\propto', '∝',
'\\prec', '≺', '\\succ', '≻', '\\preceq', '⪯', '\\succeq', '⪰',
'\\subset', '⊂', '\\supset', '⊃', '\\sqsubseteq', '⊑', '\\sqsupseteq', '⊒',
# Binary operators (medium spacing in LaTeX)
'+', '-', '\\times', '×', '\\div', '÷', '\\cdot', '·',
'\\pm', '±', '\\mp', '∓',
'\\lor', '∨', '\\land', '∧', '\\wedge', '∧', '\\vee', '∨', '\\neg', '¬',
'\\cap', '∩', '\\cup', '∪',
'\\sqcap', '⊓', '\\sqcup', '⊔', '\\uplus', '⊎',
'\\oplus', '⊕', '\\otimes', '⊗', '\\odot', '⊙', '\\ominus', '⊖', '\\oslash', '⊘',
'\\curlywedge', '⋏', '\\curlyvee', '⋎',
'\\setminus', '∖', '\\ast', '∗', '\\star', '⋆', '\\circ', '∘',
'\\bullet', '•', '\\diamond', '⋄', '\\bigtriangleup', '△', '\\bigtriangledown', '▽',
'\\triangleleft', '◃', '\\triangleright', '▹',
'\\lhd', '⊲', '\\rhd', '⊳', '\\unlhd', '⊴', '\\unrhd', '⊵',
# Punctuation with spacing
',', ';', ':',
}
# Preserve spaces before these (differentials need explicit spacing)
PRESERVE_SPACE_BEFORE = {
'dx', 'dy', 'dz', 'dt', 'du', 'dv', 'dw',
'dr', 'ds', 'da', 'db', 'dc',
'd\\theta', 'd\\phi', 'd\\psi', 'd\\alpha', 'd\\beta',
'dxdy', 'dydz', 'dxdz', # Multi-variable differentials
}
class MoVisitor(Visitor):
"""
Handles <mo> (operator) MathML elements.
This visitor converts operators to LaTeX format, handling:
- Unicode symbol mapping (× → \\times, ≤ → \\leqslant)
- Delimiter recognition (fence operators)
- Spacing control
- Subject-specific operator formatting
Examples:
Standard operators:
<mo>+</mo> → +
<mo>×</mo> → \\times
<mo>≤</mo> → \\leqslant
Delimiters:
<mo fence="true">(</mo> → \\left(
<mo fence="true">)</mo> → \\right)
Chemistry:
<mo>→</mo> → \\rightarrow (or \\xlongequal{condition})
"""
def __init__(self, strategy: Strategy):
"""
Initialize the Mo visitor.
Args:
strategy: The formatting strategy to use
"""
super().__init__(strategy)
self._converter_context = None
def set_converter_context(self, converter):
"""Set converter context for color suppression checking."""
self._converter_context = converter
def visit(self, node: ET.Element) -> str:
"""
Convert <mo> element to LaTeX.
Args:
node: The <mo> element to convert
Returns:
LaTeX string representation
Raises:
ValueError: If node is not an <mo> element
"""
if not self.can_handle(node):
raise ValueError(f"MoVisitor cannot handle element '{node.tag}'")
# Get text content
content = _text_content(node)
# Check for fence attribute FIRST (delimiter handling)
# Empty fence delimiters (like closing brace in cases) need special handling
fence = node.get('fence')
if fence == 'true':
# For fence delimiters, allow empty content (will be handled by _handle_fence)
if not content:
return self._handle_fence("", node)
# Escape LaTeX special characters before fence handling
content = escape_latex_special_chars(content)
return self._handle_fence(content, node)
# Handle empty content (non-fence operators)
if not content:
return ""
# Escape LaTeX special characters before any other processing
content = escape_latex_special_chars(content)
# Handle spaces (NBSP → ~, space → \ )
if '\xa0' in content or ' ' in content:
# If content is ONLY spaces, use context-aware filtering
if not content.strip():
# Check if this space should be preserved based on context
if self._should_preserve_space(node):
converted = convert_spaces_to_latex(content)
return self._wrap_with_color(converted, node)
else:
# Skip this space - LaTeX will handle spacing
return ""
# Content has spaces mixed with other characters
# CRITICAL FIX: Strip surrounding whitespace from operators!
# MathType sometimes outputs "<mo> < </mo>" which fails to match "<" in strategy
# and gets converted to "\ < \ " by convert_spaces_to_latex.
content = content.strip()
# If strip() resulted in empty string (should catch above, but safety)
if not content:
return ""
# Use convert_spaces_to_latex ONLY if there are still spaces INSIDE the operator
# (e.g. "mod " -> "mod" (stripped), but "a b" -> "a~b"?)
if '\xa0' in content or ' ' in content:
content = convert_spaces_to_latex(content)
# Check for stretchy attribute (affects some operators like arrows)
stretchy = node.get('stretchy')
# Delegate to strategy for subject-specific operator formatting
result = self.strategy.format_operator(content, node)
# DEBUG
if content in '()':
debug_print(f"[DEBUG MO] Operator {content!r} → {result!r}")
return self._wrap_with_color(result, node)
def _handle_fence(self, content: str, node: ET.Element) -> str:
"""
Handle fence (delimiter) operators with fence="true" attribute.
Args:
content: The delimiter character (can be empty for invisible delimiters)
node: The element node
Returns:
LaTeX delimiter wrapped in __DELIM__ markers for post-processing
Note:
Fences are marked with __DELIM__ markers to indicate they should be
paired by the delimiter pairing processor. This distinguishes them from
plain operators that happen to use the same character (e.g., | in P(A|B)
is an operator, not a fence delimiter).
Empty content is converted to "." (invisible delimiter) for cases like
piecewise functions: \left\{...\right.
"""
# Handle empty or whitespace-only delimiter
# Empty: invisible delimiter in LaTeX (e.g., piecewise functions: \left\{...\right.)
# Whitespace-only: OMML endChr/begChr with space value (e.g., m:val=" ")
# → return empty string (no delimiter), NOT a dot, to prevent extra \right.
if not content or content.strip() == "":
if not content:
# Truly empty → invisible delimiter (for cases environment etc.)
result = "__DELIM__.__DELIM__"
debug_print(f"[DEBUG MO FENCE] Empty delimiter → {result!r}")
return result
else:
# Whitespace-only → skip entirely (no delimiter needed)
debug_print(f"[DEBUG MO FENCE] Whitespace-only fence content → skip (no delimiter)")
return ""
# Map common fences to LaTeX
fence_map = {
'(': '(',
')': ')',
'[': '[',
']': ']',
'{': '\\{',
'}': '\\}',
'|': '|',
'‖': '\\|', # Double vertical bar
'⟨': '\\langle',
'⟩': '\\rangle',
'⌈': '\\lceil',
'⌉': '\\rceil',
'⌊': '\\lfloor',
'⌋': '\\rfloor',
}
latex_char = fence_map.get(content, content)
# Wrap in __DELIM__ markers so post-processor knows to add \left and \right
# ONLY delimiters with fence="true" get these markers
stretchy = node.get('stretchy')
if stretchy == 'true':
result = f"__DELIM__STRETCHY::{latex_char}__DELIM__"
else:
result = f"__DELIM__{latex_char}__DELIM__"
# Debug output
debug_print(f"[DEBUG MO FENCE] {content!r} → {result!r}")
return result
def _wrap_with_color(self, latex: str, node: ET.Element) -> str:
"""
Wrap LaTeX with color command if node has color attribute.
Args:
latex: The LaTeX string to wrap
node: The element node
Returns:
LaTeX string, optionally wrapped with color command
"""
# Use the centralized color wrapping utility with suppression support
from ..utils.color_helpers import wrap_element_with_color
# Get converter context from multiple sources (for backward compatibility):
# 1. Direct visitor context (set by legacy_adapter)
# 2. Strategy context (shared across all visitors, including child visitors)
context = getattr(self, '_converter_context', None)
if context is None and hasattr(self.strategy, '_converter_context'):
context = self.strategy._converter_context
return wrap_element_with_color(latex, node, context)
def can_handle(self, node: ET.Element) -> bool:
"""
Check if this visitor can handle the given node.
Args:
node: The MathML element to check
Returns:
True if node is an <mo> element, False otherwise
"""
from ..utils import local_name as _local
return _local(node.tag) == 'mo'
def _should_preserve_space(self, node: ET.Element) -> bool:
"""
Context-aware decision: should this space operator be preserved?
Analyzes the previous and next sibling elements to determine if the space
is redundant (LaTeX will add it automatically) or necessary.
Args:
node: The <mo> space element
Returns:
True if space should be preserved, False if it should be skipped
"""
parent = node.getparent()
if parent is None:
return False # No context, skip
siblings = list(parent)
try:
idx = siblings.index(node)
except ValueError:
return False # Can't find in parent, skip
# Rule 1: Skip trailing spaces (no next element)
if idx >= len(siblings) - 1:
return False
# Get previous and next elements
prev_elem = siblings[idx - 1] if idx > 0 else None
next_elem = siblings[idx + 1]
# Rule 2: Get LaTeX representation of previous element
if prev_elem is not None:
prev_latex = self._peek_sibling_latex(prev_elem)
# Skip after operators with built-in spacing
if prev_latex.strip() in SKIP_SPACE_AFTER:
return False
# Skip after LaTeX functions (\sin, \cos, etc.)
# Check if converted from plain text to LaTeX command
if prev_latex.startswith('\\'):
func_name = prev_latex.lstrip('\\').split()[0] if prev_latex else ''
if func_name in {'sin', 'cos', 'tan', 'log', 'ln', 'exp',
'lim', 'max', 'min', 'sup', 'inf',
'arcsin', 'arccos', 'arctan', 'sinh', 'cosh', 'tanh'}:
return False
# Rule 3: Preserve before differentials
next_latex = self._peek_sibling_latex(next_elem)
if next_latex.strip() in PRESERVE_SPACE_BEFORE:
return True
# Rule 4: Default - skip (let LaTeX handle spacing)
return False
def _peek_sibling_latex(self, elem: ET.Element) -> str:
"""
Convert a sibling element to LaTeX without full recursion.
This is a lightweight peek to check what LaTeX representation
the element would produce, used for context decisions.
Args:
elem: The sibling element to peek at
Returns:
LaTeX representation (simplified)
"""
if elem is None:
return ""
tag = _local(elem.tag)
# For simple elements, just get text content
if tag == 'mi':
text = _text_content(elem).strip()
# Check if it's a function name
if text in {'sin', 'cos', 'tan', 'log', 'ln', 'exp',
'lim', 'max', 'min', 'sup', 'inf'}:
return f'\\{text}'
return text
elif tag == 'mo':
text = _text_content(elem).strip()
# Common operator conversions
op_map = {
'×': '\\times', '÷': '\\div', '·': '\\cdot',
'≤': '\\leq', '≥': '\\geq', '≠': '\\neq',
'→': '\\rightarrow', '←': '\\leftarrow', '↔': '\\leftrightarrow',
'∨': '\\lor', '∧': '\\land', '¬': '\\neg',
'∩': '\\cap', '∪': '\\cup',
'⊥': '\\perp', '∥': '\\parallel',
'∈': '\\in', '⊆': '\\subseteq', '⊇': '\\supseteq',
}
return op_map.get(text, text)
elif tag == 'mn':
return _text_content(elem).strip()
else:
# For complex elements, return empty (conservative)
return ""
@property
def element_name(self) -> str:
"""Get the element name this visitor handles."""
return 'mo'
def __repr__(self) -> str:
"""String representation for debugging."""
return f"MoVisitor(strategy={self.strategy.name})"