How to inverse lemmatization process given a lemma and a token?

2024/10/15 21:13:41

Generally, in natural language processing, we want to get the lemma of a token.

For example, we can map 'eaten' to 'eat' using wordnet lemmatization.

Is there any tools in python that can inverse lemma to a certain form?

For example, we map 'go' to 'gone' given target form 'eaten'.

PS: Someone mentions we have to store such mappings. How to un-stem a word in Python?

Answer

Turning a base form such as a lemma into a situation-appropriate form is called realization (or "surface realization"). Example from Wikipedia:

NPPhraseSpec subject = nlgFactory.createNounPhrase("the", "woman");
subject.setPlural(true);
SPhraseSpec sentence = nlgFactory.createClause(subject, "smoke");
sentence.setFeature(Feature.NEGATED, true);
System.out.println(realiser.realiseSentence(sentence));
// output: "The women do not smoke."

Libraries for this are not as frequently used as lemmatizers, which generally means you have fewer options and are less likely to find a well developed library. The Wikipedia example is in Java because the most popular library supporting this is SimpleNLG.

A quick search found pynlg, though it doesn't seem actively maintained. Alternately you can use SimpleNLG via an HTTP JSON interface provided by the Python library nlgserv.

https://en.xdnf.cn/q/69241.html

Related Q&A

NameError: name N_TOKENS is not defined

I am new on Python and just got around to install PyCharm for Windows. Downloaded some sample code from Skype for testing their SkypeKit API. But... As soon as I hit the debug button, I get this: (I ha…

Moving Spark DataFrame from Python to Scala whithn Zeppelin

I created a spark DataFrame in a Python paragraph in Zeppelin.sqlCtx = SQLContext(sc) spDf = sqlCtx.createDataFrame(df)and df is a pandas dataframeprint(type(df)) <class pandas.core.frame.DataFrame&…

How do I efficiently do a bulk insert-or-update with SQLAlchemy?

Im using SQLAlchemy with a Postgres backend to do a bulk insert-or-update. To try to improve performance, Im attempting to commit only once every thousand rows or so:trans = engine.begin()for i, rec in…

How to pass variables from javascript to python in Jupyter?

As I understand it, I should be able to print the variable foo in the snippet below. from IPython.display import HTML HTML(<script type="text/javascript">IPython.notebook.kernel.execute…

SVR Model --Feature Scaling - Expected 2D array, got 1D array instead

I am trying to understand what is wrong with the code below. I know that the Y variable is 1D array and expected to be 2D array and need to reshape the structure but that code was working previously fi…

How to find the version of jupyter notebook from within the notebook

I wish to return the version of Jupyter Notebook from within a cell of a notebook. For example, to get the python version, I run: from platform import python_version python_version()or to get the panda…

Python logging - multiple modules

Im working on a small python project that has the following structure -project -- logs-- project__init.py__classA.pyclassB.pyutils.py-- main.pyIve set up the logging configuration in __init.py__ under …

Can you search backwards from an offset using a Python regular expression?

Given a string, and a character offset within that string, can I search backwards using a Python regular expression?The actual problem Im trying to solve is to get a matching phrase at a particular of…

Django AttributeError: Form object has no attribute _errors

Im overriding the init method in my form andthis is now returning an error TransactionForm object has no attribute _errors. I would expect this to work because Ive included super in my init, however pe…

Add new keys to a dictionary while incrementing existing values

I am processing a CSV file and counting the unique values of column 4. So far I have coded this three ways. One uses "if key in dictionary", the second traps the KeyError and the third uses &…