Find Jobs
Hire Freelancers

Analyze some Data - Parse raw log data into CSV format.

$10-30 USD

Completed
Posted almost 9 years ago

$10-30 USD

Paid on delivery
I have a raw file full of entries about 3GB in size. I have tried using log parser however this is not a standardised file format so it must be formatted. Space " " is used as a delimited however some fields have countries made of two worlds "United Kingdom" which will contain a space. I need a simple script (Visual Basic Script, Batch, PowerShell or whichever) that will be able to go through the entire log file, determine the fields and generate me a CSV file with the data in the correct fields. Here is a sample file. Each paragraph represents an entry row. Some entries do not have the same fields, some entries have fields in different orders. May 11 00:00:00 [login to view URL] date=2015-05-11 time=00:00:00 devname=DEMO-unit1 devid=FG200B3912608545 logid=0000000013 type=traffic subtype=forward level=notice vd=root srcip=[login to view URL] srcport=50427 srcintf="port15" dstip=[login to view URL] dstport=53 dstintf="port16" sessionid=756474921 status=accept policyid=1 dstcountry="Ireland" srccountry="Reserved" trandisp=snat transip=[login to view URL] transport=50427 service=DNS proto=17 duration=180 sentbyte=81 rcvdbyte=128 sentpkt=1 rcvdpkt=1 May 11 00:00:05 [login to view URL] date=2015-05-11 time=00:00:05 devname=DEMO-unit1 devid=FG200B3912608545 logid=0001000014 type=traffic subtype=local level=notice vd=root srcip=[login to view URL] srcport=55906 srcintf="port15" dstip=[login to view URL] dstport=161 dstintf="root" sessionid=756475176 status=accept policyid=0 dstcountry="Reserved" srccountry="Reserved" trandisp=noop service=SNMP proto=17 app=SNMP duration=180 sentbyte=180 rcvdbyte=349 sentpkt=1 rcvdpkt=2 May 11 00:00:10 [login to view URL] date=2015-05-11 time=00:00:10 devname=DEMO-unit1 devid=FG200B3912608545 logid=0000000013 type=traffic subtype=forward level=notice vd=root srcip=[login to view URL] srcport=137 srcintf="port15" dstip=[login to view URL] dstport=137 dstintf="port15" sessionid=756485114 status=deny policyid=0 dstcountry="Reserved" srccountry="Reserved" trandisp=noop service=137/udp proto=17 duration=0 sentbyte=0 rcvdbyte=0 May 11 00:00:19 [login to view URL] date=2015-05-11 time=00:00:19 devname=DEMO-unit1 devid=FG200B3912608545 logid=0000000013 type=traffic subtype=forward level=notice vd=root srcip=[login to view URL] srcport=59107 srcintf="port15" dstip=[login to view URL] dstport=111 dstintf="Manchester-p1" sessionid=756476011 status=accept policyid=190 dstcountry="Reserved" srccountry="Reserved" trandisp=noop service=ONC-RPC proto=17 duration=180 sentbyte=84 rcvdbyte=0 sentpkt=1 rcvdpkt=0 vpn="Manchester-p1" vpntype=ipsec-static The script should try and automatically detect the fields without inputting them manually. The end results should be something like this. month ¦ day ¦ hour ¦ ip ¦ date ¦ time May ¦ 11 ¦ 00:00:10 ¦ [login to view URL] ¦ 2015-05-11 ¦ 00:00:19 May ¦ 11 ¦ 00:00:10 ¦ ¦ 2015-05-11 ¦ 00:00:19 That should be the end result. The second result has no IP. This means that the actual entry in the raw log file does not have the ip entry. Most of the fields are identifiable as "field=data".
Project ID: 7723940

About the project

7 proposals
Remote project
Active 9 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
I'm expert in data processing / parsing that's why I'm sure you'll be impressed with my work. I can create log parser you want (your log file => CSV) in less than a day and I can offer you best price here. But first I need to have a look at actual file (just a sample like post). Because some formatting may be changed if you post it as a project description. So can you post it as separate file? Also I need to have a look at the line with "United Kingdom". Thanks. Roman
$30 USD in 1 day
5.0 (144 reviews)
6.1
6.1
7 freelancers are bidding on average $38 USD for this job
User Avatar
Hello Sr., how are you ? Jorge is my name and I have wide experience doing scrapes. You can visit my Portfolio to know my last jobs regarding these skills, the reviews received from clients and as you can see, a higher percentage of jobs completed. i'm interested to help you on this particular project. What I prefer is to discuss with you more in detail all your terms and I'll give you a better proposal. No more Sr.; contact me if you'd like it. Regards. Jorge. P.S.: I will not accept any award before to discuss with you all the details needed.
$55 USD in 2 days
5.0 (111 reviews)
6.6
6.6
User Avatar
hi, I've read your project description, and I can come out a Perl parser to format the log file into csv file format. I'm ready to start anytime now.
$55 USD in 1 day
5.0 (10 reviews)
4.5
4.5
User Avatar
If you don't mind, It's quite easy to do the parsing, I can create a simple tool app using Delphi.
$30 USD in 2 days
5.0 (6 reviews)
3.7
3.7
User Avatar
I can create a powershell gui application to do it activity. I will also provide support to make changes in the app when you find any bug and add few more features in it
$15 USD in 2 days
5.0 (1 review)
1.7
1.7
User Avatar
Hi, I can do this because I did lots of this when I worked for telecom. I will require $30 sent to my paypal or to bank acount in Canada. If it is one time job, then just send me the file. Result should be ready by monday 25. Best regards, Gleb
$25 USD in 3 days
0.0 (0 reviews)
0.0
0.0
User Avatar
A proposal has not yet been provided
$55 USD in 2 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED KINGDOM
London, United Kingdom
5.0
2
Payment method verified
Member since May 22, 2015

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.