How to Import and Parse CSV Files into MongoDB Using Node.js, Express, and Multer
Handling bulk data uploads is a frequent requirement in enterprise applications and student record management systems. If you need a reliable way to upload spreadsheet data, parse rows, and store records directly in a database, learning how to process CSV files is essential.
In this comprehensive, step-by-step tutorial, you will learn how to import and parse CSV files into MongoDB using Node.js, Express, Multer, and the csvtojson package. We will walk through setting up your project folders, configuring Multer disk storage, converting CSV rows into structured JSON arrays, and rendering records dynamically on an EJS frontend table.
Project Structure
Organize your project files cleanly with the following architecture:
csv/ ├── models/ │ └── csv.js ├── public/ │ └── uploads/ ├── views/ │ └── demo.ejs ├── app.js └── package.jsonStep 1: Set Up Your Project Folder
Open your command prompt or terminal and create your working directory:
# Create a new directory and move into it
mkdir csv
cd csv
Step 2: Initialize Node.js and Install Dependencies
Initialize your Node.js project environment automatically:
npm init -y
Next, install all the necessary packages for file uploads (multer), CSV parsing (csvtojson), template rendering (ejs), database modeling (mongoose), and routing (express & body-parser):
npm install csvtojson express ejs mongoose multer body-parser
Create the required folder structure in your project root:
models/public/uploads/views/
Step 3: Define the MongoDB Schema(models/csv.js)
Create a new file inside the models folder named csv.js to define your Mongoose student records schema:
var mongoose = require('mongoose');
var csvSchema = new mongoose.Schema({ FirstName: { type: String }, LastName: { type: String }, SSN: { type: String }, Test1: { type: Number }, Test2: { type: Number }, Test3: { type: Number }, Test4: { type: Number }, Final: { type: Number }, Grade: { type: String }});
module.exports = mongoose.model('studentsrecords', csvSchema);
Step 4: Create the Frontend View (views/demo.js)
Create a file inside the views folder named demo.ejs. This file provides the file upload form and checks if records exist to render them in a responsive table:
<html lang="en"><head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta http-equiv="X-UA-Compatible" content="ie=edge"> <title>Document</title> <link rel="stylesheet" href="/css/bootstrap.min.css"></head><body> <nav class="navbar navbar-expand-lg navbar-dark bg-primary fixed-top"> <a class="navbar-brand" href="#">CsvToMongo</a> </nav> <div class="container"> <div class=" nav justify-content-center" style="margin-top:100px;"> <div class="card border-warning mb-3 " style="max-width: 20rem;"> <div class="card-header"><h5>Upload csv file</h5></div> <div class="card-body"> <form action="/" method="post" enctype="multipart/form-data"> <input type="file" name="csv"><br><br> <div class="text-center"><button type="submit" class="btn btn-lg btn-primary">submit</button></div>
</form> </div> </div> </div><br> <%if(data){%> <div> <table class="table table-hover table-responsive table-stripped nav justify-content-center" style="width: auto" >
<thead> <tr class="bg-primary"> <th>S.no</th> <th style="padding-right: 1em">LastName</th> <th style="padding-right: 1em">FirstName</th> <th style="padding-right:2em;padding-left:2em;">SSN</th> <th>Test1</th> <th>Test2</th> <th>Test3</th> <th>Test4</th> <th>Final</th> <th>Grade</th> </tr> </thead> <tbody style="overflow-x: scroll; height:350px;" class="table-bordered"> <%for(var i=0;i< data.length;i++){%> <tr class="text-center"> <td ><%= i+1%></td> <td style="padding-right: 1em"><%= data[i].LastName%></td> <td style="padding-left: 1em;"><%= data[i].FirstName%></td> <td style="padding-right:1em;padding-left:1em;"><%= data[i].SSN%></td> <td style="padding-left: 1em"><%= data[i].Test1%></td> <td style="padding-left: 1em"><%= data[i].Test2%></td> <td style="padding-left: 1em"><%= data[i].Test3%></td> <td style="padding-left: 1.2em"><%= data[i].Test4%></td> <td style="padding-left: 1.2em"><%= data[i].Final%></td> <td style="padding-left: 1.2em"><%= data[i].Grade%></td> </tr> <%}%> </tbody> </table> </div> <%}%><br></body></html>
Step 5: Configure Entry Point (app.js)
Create your main application entry point named app.js in the project root. Open your package.json file and under "scripts" add "start": "node app.js".
var express = require('express');var mongoose = require('mongoose');var multer = require('multer');var path = require('path');var csvModel = require('./models/csv');var csv = require('csvtojson');var bodyParser = require('body-parser');
var storage = multer.diskStorage({ destination:(req,file,cb)=>{ cb(null,'./public/uploads'); }, filename:(req,file,cb)=>{ cb(null,file.originalname); }});
var uploads = multer({storage:storage});
//connect to dbmongoose.connect('mongodb://localhost:27017/csvdemos',{useNewUrlParser:true}).then(()=>console.log('connected to db')).catch((err)=>console.log(err))
//init appvar app = express();
//set the template engineapp.set('view engine','ejs');
//fetch data from the requestapp.use(bodyParser.urlencoded({extended:false}));
//static folderapp.use(express.static(path.resolve(__dirname,'public')));
//default pageloadapp.get('/',(req,res)=>{ csvModel.find((err,data)=>{ if(err){ console.log(err); }else{ if(data!=''){ res.render('demo',{data:data}); }else{ res.render('demo',{data:''}); } } });});
var temp ;
app.post('/',uploads.single('csv'),(req,res)=>{
//convert csvfile to jsonArray
csv().fromFile(req.file.path).then((jsonObj)=>{ console.log(jsonObj); for(var x=0;x<jsonObj;x++){ temp = parseFloat(jsonObj[x].Test1) jsonObj[x].Test1 = temp; temp = parseFloat(jsonObj[x].Test2) jsonObj[x].Test2 = temp; temp = parseFloat(jsonObj[x].Test3) jsonObj[x].Test3 = temp; temp = parseFloat(jsonObj[x].Test4) jsonObj[x].Test4 = temp; temp = parseFloat(jsonObj[x].Final) jsonObj[x].Final = temp; } csvModel.insertMany(jsonObj,(err,data)=>{ if(err){ console.log(err); }else{ res.redirect('/'); } }); });});
//assign portvar port = process.env.PORT || 3000;app.listen(port,()=>console.log('server run at port '+port));
Conclusion & Resources
By combining Multer, csvtojson, and Mongoose, you have successfully implemented an automated data pipeline to parse CSV files and bulk-insert records into MongoDB with Node.js.
SourceCode: Download Code From here
Video Tutorial: Watch Video Tutorial
Comments
Post a Comment