Where to harvest program lists for editor application

[aplok]

Re: Where to harvest program lists for editor application

ok, c# is like other stuff billshit did it: nice outside, dirty in

my message is simple:

you will gain with some simple training (one weekend)
a simple intro tuto
with perl

parsing wiil appear clearly

try it ! and you will see ...

[sjamesamuel]

Re: Where to harvest program lists for editor application

- Not necessarily
ok ! but only after a demo !

You can ******: c# regex. There are tons of simple examples.

so, please, submit us your one-liner C-jail solution to the challenge ...

Sure, using Linq:

var result= s.Split(" ;.,".ToCharArray(), StringSplitOptions.RemoveEmptyEntries).Distinct().OrderBy(x => x).ToList();

I think it looks even better

[aplok]

Re: Where to harvest program lists for editor application

- You can ******: c# regex. There are tons of simple examples.
i know, and this is precisely the topic of the challenge

var result= s.Split(" ;.,".ToCharArray(), StringSplitOptions.RemoveEmptyEntries).Distinct(). OrderBy(x => x).ToList();

there are some missing parts in this c# program
and therefore the presented solution not fully operational
-> it fails the challenge

- I think it looks even better
if so is ok for your ...

[sjamesamuel]

Re: Where to harvest program lists for editor application

there are some missing parts in this c# program
and therefore the presented solution not fully operational
-> it fails the challenge

If you mean displaying the output, here (no need for the split options):

s.Split(" ;.,".ToCharArray()).Distinct().OrderBy(x => x).ToList().ForEach(x => Console.WriteLine(x));

Compared to:

cat text.a |perl -ne 'BEGIN{$a=""}{chomp($_); $a.=$_}END{$a=~s/[ \.,]/\n;/g; print sort( split(/;/,$a)) }'


Anyway the required program is a simple application on any language and I think Teranoz should write it using regular expressions on any language.

The KOS format can be very easily parsed using regular expressions

1=10714,H,22000,56,DVB-S,QPSK
2=10729,V,22000,56,DVB-S,QPSK
3=10744,H,22000,56,DVB-S,QPSK
4=10758,V,22000,56,DVB-S,QPSK

If you need any help you can ask here.

[Teranoz]

Re: Where to harvest program lists for editor application

if (charstospliton == (" ;.,").Only)
{
Kingofsat.parsing.difficult = false;
}
else {
Perl.useful.for.me = false;
}

[sjamesamuel]

Re: Where to harvest program lists for editor application

You don't have to do any char splitting (this was aplok's challenge)

Here's the regex for you: (It may need some adjustments for other special cases)

(\d+)=(\d+),([HV]),(\d+),(\d+),(.+?),(.+)

It has seven capturing groups (brackets). Each will capture one of the seven fields in the code below. All what you need to do is perform the matching on the downloaded file, and to find how to use regexes in your language of choice.

3=10744,H,22000,56,DVB-S,QPSK
10=10847,V,23000,23,S2,8PSK

[aplok]

Re: Where to harvest program lists for editor application

@sjamesamuel
- the need to write the output
yes

and, last but not least
- read the input file
- includes's
- compile
- run
- ... and all those details which convert nice but partial evidences
into an operational liner

sorry, but your proposed code:
s.Split(" ;.,".ToCharArray()).Distinct().OrderBy(x => x).ToList()[B].ForEach(x => Console.WriteLine(x))

is still not operational
-> it fails the challenge


as an example, to parse this:

1=10714,H,22000,56,DVB-S,QPSK
2=10729,V,22000,56,DVB-S,QPSK
3=10744,H,22000,56,DVB-S,QPSK
4=10758,V,22000,56,DVB-S,QPSK


you can try this:

cat a.txt |perl -ne 's/=/,/; @a=split(/,/); print "$a[1]::$a[2]::$a[3]\n"'

[sjamesamuel]

Re: Where to harvest program lists for editor application

- read the input file
- includes's
- compile
- run

To read the input file, in the same statement:

File.ReadAllText("a.txt").Split(" ;.,".ToCharArray()).Distinct().OrderBy(x => x).ToList().ForEach(x => Console.WriteLine(x));

There are no "includes" in .NET. There are "using" statements, which can be replaced by the namespaces (System.IO.File).. But that's trivial.
And you can't count compile and run as missing details. Obviously any program will have them.

ps The c# solution has only one statement.
Yours has a couple of statements but you only put them on the same line.

[aplok]

Re: Where to harvest program lists for editor application

@jamesamuel
- .. which can be replaced by the namespaces (System.IO.File).. But that's trivial
trivial, ...
but not obvious at all to somebody,
out of the .net dreamland,
and willing to test it

by the way can you complete this liner with this namespaces (System.IO.File) instructions, thx

- And you can't count compile and run as missing details.
... but still, this (compile) has to be processed successfully, isn't ?

- Obviously any program will have them.
not with perl,
perl code is a script executed by perl parser:
so no compilation needed

- ps The c# solution has only one statement.
Yours has a couple of statements but you only put them on the same line.
yes, that is the philosophy of liners: one line code do the job of a program (like C)

[sjamesamuel]

Re: Where to harvest program lists for editor application

@jamesamuel
- .. which can be replaced by the namespaces (System.IO.File).. But that's trivial
trivial, ...
but not obvious at all to somebody,
out of the .net dreamland,
and willing to test it

by the way can you complete this liner with this namespaces (System.IO.File) instructions, thx

Each function belongs to a namespace. So you can either add this at the top of the program (they are added by default)

using System;
using System.IO;
using System.Linq;

and use File .. directly, or remove the using statement, and replace "File" with "System.IO.File". You'll just add the name space before the function name.

- And you can't count compile and run as missing details.
... but still, this (compile) has to be processed successfully, isn't ?

Of course like any code has to be written correctly. Once you compile and build, you'll only run the executable, not compile every time.

- ps The c# solution has only one statement.
Yours has a couple of statements but you only put them on the same line.
yes, that is the philosophy of liners: one line code do the job of a program (like C)

I can also write 10 statements of c# on one line (each terminated by a semicolon).

I think we have diverged a lot from the original topic

[aplok]

Re: Where to harvest program lists for editor application

if (charstospliton == (" ;.,").Only)
{Kingofsat.parsing.difficult = false;}
else {Perl.useful.for.me = false;}

i wonder the parsing technology you will prefer to achieve your goal

can you share your experience here ?

thx

[Teranoz]

Re: Where to harvest program lists for editor application

Ini needs no parsing :

private String _GetIniFileString(string category, string key, string defaultValue)
{
string iniFile = this.IniFile;
if (iniFile == "")
{
return "";
}
else
{
string returnString = new string(' ', 1024);
GetPrivateProfileString(category, key, defaultValue, returnString, 1024, iniFile);
return returnString.Split('\0')[0];
}
}

for the parsing I use atm just local saved html that I load until it works without problems.
Part of the parsing :

sats :

Inhoud = Inhoud.Replace("<TD", "<td");
String[] htmlTransponders = Inhoud.Split(new string[] { "class=\"frq\"" }, StringSplitOptions.RemoveEmptyEntries);
for (int i = 1; i < htmlTransponders.Length; i++)
{
string[] htmFL = htmlTransponders[i].Split(new string[] { "class=\"fl\"" }, StringSplitOptions.RemoveEmptyEntries);
string[] tpInfo = htmFL[0].Split(new string[] { "<td" }, StringSplitOptions.RemoveEmptyEntries);
cTransponderItem Tp = new cTransponderItem();
int yellow = tpInfo[1].IndexOf("yellow") + 8;
int endPos = tpInfo[1].IndexOf("<", yellow);
String Positie = tpInfo[1].Substring(yellow, endPos - yellow).Replace("&deg;", "");
decTemp = csUtils.StringToDecimal(Positie.Substring(0, Positie.Length - 1));
if (Positie.IndexOf("W") > -1)
{
decTemp = decTemp * -1;
}
Sat.longitude = decTemp;

Part of the program parsing :

//Dat was de regel met transponder/sat info. Nu volgen de zenders
String[] TrxInfoRow = htmFL[1].Split(new String[] { "<tr" }, StringSplitOptions.RemoveEmptyEntries);

for (int j = 1; j < TrxInfoRow.Length; j++)
{
//Elke regel is een TR met zenderinfo verdeeld in meerdere TD's
if (TrxInfoRow[j].IndexOf("/zap.jpg") < 0 && TrxInfoRow[j].IndexOf("=\"feed\"") < 0 && TrxInfoRow[j].IndexOf("/data.gif") < 0)
{
String[] TrxInfoTD = TrxInfoRow[j].Split(new String[] { "<td" }, StringSplitOptions.RemoveEmptyEntries);
if (TrxInfoRow[j].IndexOf("radio.gif") > 0)
{
/*{string[13]}
[0]: " bgcolor=\"white\">"
[1]: " class=\"r\"></td>"
[2]: " WIDTH=\"3%\"><img src=\"/radio.gif\" border=0></TD>\r\n"
[3]: " WIDTH=\"36%\" colspan=3>RTE Radio 1</TD>\r\n"
[4]: " WIDTH=\"12%\"><a class=\"bq\" href=\"pack-freesat.php\">Freesat</a><br><a class=\"bq\" href=\"pack-skydigital.php\">Sky Digital</a></TD>\r\n"
[5]: " WIDTH=\"10%\">FTA</TD>\r\n"
[6]: " WIDTH=\"5%\" class=\"s\">9611</TD>\r\n"
[7]: " WIDTH=\"5%\" >&nbsp;</TD>\r\n"
[8]: " WIDTH=\"8%\">2318<a title=\"English\"> <font color=\"blue\">eng</font></a>&nbsp;</TD>\r\n"
[9]: " WIDTH=\"4%\">268&nbsp;</TD>\r\n"
[10]: " WIDTH=\"4%\">2305&nbsp;</TD>\r\n"
[11]: " WIDTH=\"4%\">&nbsp;</TD>\r\n"
[12]: " WIDTH=\"10%\"><a >2012-02-18</a></td></tr> "*/
}
if (TrxInfoTD.Length > 11)
{
cProgram Zender = new cProgram();
int Delta = 0;
if (TrxInfoRow[j].IndexOf("radio.gif") > 0)
{
Zender.Soort = "RADIO";
Delta = 2;
if (TrxInfoTD.Length == 15) Delta = 0;
} else
{
Zender.Soort = "TV";
}
Zender.UTF8Name = StripHTML(TrxInfoTD[3]).ToCharArray();
Zender.name = StripHTML(TrxInfoTD[3]);

if (Delta == 0)
{
Zender.Land = StripHTML(TrxInfoTD[4]);
Zender.Categorie = StripHTML(TrxInfoTD[5]);
}
Zender.provider = StripHTML(TrxInfoTD[6 - Delta].Replace("<br />", ", ").Replace("<br>", ", "));
if(Zender.provider.IndexOf(',')>0) Zender.provider = Zender.provider.Substring(0,Zender.provider.IndexO f(','));//Alleen de eerste
Zender.provider = Zender.provider.Replace(" ", " ");
if (Zender.provider == "-") Zender.provider = "";
Zender.encrypt = StripHTML(TrxInfoTD[7 - Delta].Replace("<br />", ", ").Replace("<br>", ", "));
Zender.service_id = csUtils.StringToInt32(StripHTMLNumbersToFirstBreak (TrxInfoTD[8 - Delta]));
if (Delta == 0 && Zender.Soort == "TV")
{
Zender.video_pid = csUtils.StringToInt32(StripHTMLNumbersToFirstBreak (TrxInfoTD[9]));
Zender.hd = TrxInfoTD[9].IndexOf("hd.gif") > 0;
}
String AudioInfo = TrxInfoTD[10 - Delta];
Zender.audio_pid = csUtils.StringToInt32(StripHTMLNumbersToFirstBreak (AudioInfo));

For the stripping of HTML I use the following function :

private static String StripHTML(String HTML)
{
String Res = "";
Boolean BeginTag = false;
Boolean EindTag = true;
Boolean EindQuote = true;
if (HTML[0] != '<') HTML = "<" + HTML;
for (int i = 0; i < HTML.Length; i++)
{
if (HTML[i] == '"')
{
if (EindQuote == true)
{
EindQuote = false;
}
else {
EindQuote = true;
}
}
else if (HTML[i] == '<' && EindTag == true && EindQuote == true)
{
BeginTag = true;
}
else if (HTML[i] == '>' && BeginTag == true && EindQuote == true)
{
EindTag = true;
BeginTag = false;
}
else if (EindTag == true && BeginTag == false && EindQuote == true)
{
Res += HTML[i];
}
}
Res = Res.Replace("&nbsp;", " ");
Res = Res.Replace("\r", "");
Res = Res.Replace("\n", "");
return Res.Trim();
}

and I use some more or less comparable functions like the StripHTMLNumbersToFirstBreak what turned out to be just transformated into a function that does more something like StripHTMLNumbersUntilSomethingElseThanANumberAppea rs

[aplok]

Re: Where to harvest program lists for editor application

congratulations,
nice job

[Teranoz]

Re: Where to harvest program lists for editor application

The current version now stores the data into MSSQL 2008 express so I can do queries with joins and distincts and so to see if there are any problems.
I also use tables for my existing data to see where the favourites go (amiko uses the internal generated primary key of the program (no name, sat of tp ref) as only reference to the favourite)
Keeping it in that DB would mean it might not be suitable for the no-nerds. But I keep on experimenting with it because it still is fun.

[aplok]

Re: Where to harvest program lists for editor application

The current version now stores the data into MSSQL 2008 express so I can do queries ...

i am happy you finally succeeded to parse such an html file
with or without regexps

and that you see, by yourself
that, for this goal, parsing a html or a xml
should not make big difference
for a programmer

this is one reward of this challenge ...