I have the following lambda function as part of Kinesis firehose record transformation which transforms msgpack record from the kinesis input stream to json.

Lambda Runtime: python 3.6

from __future__ import print_functionimport base64
import msgpack
import json
print('Loading function')def lambda_handler(event, context):output = []for record in event['records']:payload = msgpack.unpackb(base64.b64decode(record['data']), raw=False)# Do custom processing on the payload hereoutput_record = {'recordId': record['recordId'],'result': 'Ok','data': json.dumps(payload, ensure_ascii=False).encode('utf8')}output.append(output_record)print('Successfully processed {} records.'.format(len(event['records'])))return {'records': output}

But lambda throwing the following error:

An error occurred during JSON serialization of response: b'
{"id": "d23fd47f-3a62-4383-bcb3-abdb913ea572","timestamp": 1526358140730,"message": "Hello World"
}
' is not JSON serializable
Traceback (most recent call last):
File "/var/lang/lib/python3.6/json/__init__.py", line 238, in dumps
**kw).encode(obj)
File "/var/lang/lib/python3.6/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/var/lang/lib/python3.6/json/encoder.py", line 257, in iterencodereturn _iterencode(o, 0)
File "/var/runtime/awslambda/bootstrap.py", line 110, in 
decimal_serializer
raise TypeError(repr(o) + " is not JSON serializable")

Am I doing anything wrong?

I was able to fix the issue.

Here is the code which worked for me.

from __future__ import print_functionimport base64
import msgpack
import jsonprint('Loading function')def lambda_handler(event, context):output = []for record in event['records']:payload = msgpack.unpackb(base64.b64decode(record['data']), raw=False)# Do custom processing on the payload hereoutput_record = {'recordId': record['recordId'],'result': 'Ok','data': base64.b64encode(json.dumps(payload).encode('utf-8') + b'\n').decode('utf-8')}output.append(output_record)print('Successfully processed {} records.'.format(len(event['records'])))return {'records': output}