- Document Number:
20240096311
- Appl. No:
18/204585
- Application Filed:
June 01, 2023
- نبذة مختصرة :
The disclosure relates to a method for training a machine reading comprehension model, a computer-readable recording medium storing the method, and a question answering system using a trained machine reading comprehension model. The method for training a machine reading comprehension model includes: obtaining features of input sentences; generating an input sequence including a plurality of tokens based on at least one feature among the obtained features; embedding the input sequence using a pre-trained language model; inputting an output embedding, obtained by embedding the input sequence, to a first encoding layer to perform word-level encoding; and inputting a portion of hidden states of the first encoding layer to a second encoding layer to perform sentence-level encoding.
- Assignees:
Hyundai Motor Company (Seoul, KR), KIA CORPORATION (Seoul, KR)
- Claim:
1. A method for training a machine reading comprehension model, the method comprising: obtaining features of input sentences; generating an input sequence including a plurality of tokens based on at least one feature among the obtained features; embedding the input sequence using a pre-trained language model; inputting an output embedding, obtained by embedding the input sequence, to a first encoding layer to perform word-level encoding; and inputting a portion of hidden states of the first encoding layer to a second encoding layer to perform sentence-level encoding.
- Claim:
2. The method of claim 1, wherein the input sentences include a question sentence and sentences included in a passage including an answer span to the question sentence.
- Claim:
3. The method of claim 2, wherein an output of the first encoding layer includes information about a starting position and an ending position of the answer span among the input sentences.
- Claim:
4. The method of claim 3, wherein an output of the second encoding layer includes information about a position of a sentence including the answer span among the input sentences.
- Claim:
5. The method of claim 4, further comprising: calculating a first loss value for the starting position of the answer span and a second loss value for the ending position of the answer span; and calculating a third loss value for the position of the sentence including the answer span.
- Claim:
6. The method of claim 5, further comprising: adjusting weights of the first encoding layer, the second encoding layer, and the pre-trained language model based on the first loss value, the second loss value and the third loss value.
- Claim:
7. The method of claim 2, wherein the embedding comprises concatenating at least one feature of the features of the input sentences to the output embedding output from the pre-trained language model.
- Claim:
8. The method of claim 1, wherein a hidden state for tokens distinguishing the input sentences among the hidden states of the first encoding layer is input to the second encoding layer.
- Claim:
9. The method of claim 7, wherein the at least one feature concatenated to the output embedding includes at least one of a category classification result or a syllable separation result of the input sentences.
- Claim:
10. The method of claim 1, wherein the embedding comprises generating an input embedding input to the pre-trained language model by summing a token embedding, a segment embedding and a position embedding for the input sequence.
- Claim:
11. A computer-readable recording medium storing a program for implementing a method for training a machine reading comprehension model, the method comprising: generating an input sequence including a plurality of tokens based on at least one feature among features of input sentences; embedding the input sequence using a pre-trained language model; inputting an output embedding, obtained by embedding the input sequence, to a first encoding layer to perform word-level encoding; and inputting a portion of hidden states of the first encoding layer to a second encoding layer to perform sentence-level encoding.
- Claim:
12. The computer-readable recording medium of claim 11, wherein the input sentences include a question sentence and sentences included in a passage including an answer span to the question sentence.
- Claim:
13. The computer-readable recording medium of claim 12, wherein an output of the first encoding layer includes information about a starting position and an ending position of the answer span among the input sentences.
- Claim:
14. The computer-readable recording medium of claim 13, wherein an output of the second encoding layer includes information about a position of a sentence including the answer span among the input sentences.
- Claim:
15. The computer-readable recording medium of claim 14, further comprising: calculating a first loss value for the starting position of the answer span and a second loss value for the ending position of the answer span; and calculating a third loss value for the position of the sentence including the answer span.
- Claim:
16. The computer-readable recording medium of claim 15, further comprising: adjusting weights of the first encoding layer, the second encoding layer, and the pre-trained language model based on the first loss value, the second loss value and the third loss value.
- Claim:
17. The computer-readable recording medium of claim 12, wherein the embedding comprises concatenating at least one feature of the features of the input sentences to the output embedding output from the pre-trained language model.
- Claim:
18. The computer-readable recording medium of claim 11, wherein a hidden state for tokens distinguishing the input sentences among the hidden states of the first encoding layer is input to the second encoding layer.
- Claim:
19. The computer-readable recording medium of claim 17, wherein the at least one feature concatenated to the output embedding includes at least one of a category classification result or a syllable separation result of the input sentences.
- Claim:
20. The computer-readable recording medium of claim 11, wherein the embedding comprises generating an input embedding input to the pre-trained language model by summing a token embedding, a segment embedding and a position embedding for the input sequence.
- Claim:
21. A question answering system, comprising: a communication module configured to receive a question sentence of a user; and a question answering module configured to search for an answer span to the question sentence using a trained machine reading comprehension model; wherein the trained machine reading comprehension model comprises: an embedding layer configured to embed an input sequence corresponding to the question sentence using a pre-trained language model; a first encoding layer configured to perform word-level encoding on an output embedding obtained by embedding the input sequence; and a second encoding layer configured to perform sentence-level encoding on a portion of hidden states of the first encoding layer.
- Claim:
22. The question answering system of claim 21, wherein the question answering module is configured to generate the input sequence by preprocessing the question sentence and input sentences including sentences of a passage related to the question sentence.
- Claim:
23. The question answering system of claim 22, wherein an output of the first encoding layer includes information about a starting position and an ending position of the answer span to the question sentence among the input sentences.
- Claim:
24. The question answering system of claim 23, wherein an output of the second encoding layer includes information about a position of a sentence including the answer span among the input sentences.
- Claim:
25. The question answering system of claim 22, wherein at least one feature of the features of the input sentences is concatenated to the output embedding and is input to the first encoding layer.
- Claim:
26. The question answering system of claim 22, wherein a hidden state for tokens distinguishing the input sentences among the hidden states of the first encoding layer is input to the second encoding layer.
- Claim:
27. The question answering system of claim 25, wherein the at least one feature concatenated to the output embedding includes at least one of a category classification result or a syllable separation result of the input sentences.
- Claim:
28. The question answering system of claim 21, wherein the question answering module is configured to generate an input embedding input to the pre-trained language model by summing a token embedding, a segment embedding and a position embedding for the input sequence.
- Current International Class:
10; 10; 10; 10; 10
- الرقم المعرف:
edspap.20240096311
Processing Request
No Comments.