Manipulating DoclingDocument Elements (e.g. remove text before exporting)

[deruli79]

Hello,

I want to add or remove Items from a DoclingDocument but do not know if this is supported or how to achieve this

Steps:

1. Convert PDF to DoclingDocument using DocumentConverter works great.
2. print_element_tree works great() as well as iterate_items()
3. During iterate_items() I want to remove specifics items
   -> I tried to remove elements from document.texts but this breaks the document when I try to iterate_items() again

Is there a supported way to remove_text similar to add_text method?

Thanks

[Greenheart]

My project has a similar need to process the DoclingDocument. I solved it like this:

https://github.com/Klimatbyran/garbo/blob/2ad225f49d1e69899a3b7c2f3d677efe31995454/pdf2markdown/src/parse_pdf.py#L48-L82

[jherrmann]

I am new to this great library and face a similar issue. I want to get rid of some text by label them as page-header in the DoclingDocument object. The labeling works, but the headers are still shown in export to markdown. I modified the document like this:

    for item in conv_result.document.iterate_items():
        if isinstance(item[0], DocItem):
            docitem: DocItem = item[0]
            for prov_item in docitem.prov:
                if prov_item.bbox.b > 783: # everything above 783 is header
                    docitem.label = DocItemLabel.PAGE_HEADER

@deruli79 did you remove the text also from all parents? like conv_result.document.body.children.remove(docitem.get_ref())

[deruli79]

@Greenheart Thanks for the reference, I will look into your approach.

@jherrmann
conv_result.document.body.children.remove(docitem.get_ref()) in fact did improve the behavior. If I only remove an item from the body I can use iterate_items() afterwards.

If I delete the item in addition from conv_result.document.texts (which in my opinion should be done) and try to iterate I get a:
File "/usr/local/lib/python3.12/site-packages/docling_core/types/doc/document.py", line 433, in resolve\n obj = doc.getattribute(path)[index]\n ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^\nIndexError: list index out of range

In general I understand that there is no documented way (yet) to remove docitems from a Document. I am contemplating to build a DoclingDocument from scratch and use add_ methods based on the conv_result.document.
But this is much more involved.

[deruli79]

Alright, for the moment I settled with the following approach inspired by @jherrmann

1. I use iterate_items() and collect all DocItems to be removed in a list (I use a bunch of regex filters here).
2. Call following function

def remove_items_from_document(items: list[DocItem], document: DoclingDocument):
    """
    Remove Items from Document
    """

    #Iterate over items which should be removed
    for item_remove in items:

        #Resolve parent for this item
        parent_remove = item_remove.parent.resolve(document)

        #Remove from Parent
        if isinstance(parent_remove, NodeItem):
            print(f'Removing Item from Document with cref: {item_remove.self_ref}')
            parent_remove.children.remove(item_remove.get_ref())

Works well so far.

[deruli79]

This approach seems to work well