=== delcoyot1 is now known as delcoyote === raju is now known as ubucop === bulldog98_ is now known as bulldog98 [15:52] test [15:54] test passed [16:19] when does the workshop begin [16:19] in 40 minutes [16:20] oh, ok [16:20] thanks === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || Current Session: Ubuntu Algorithm Class - Instructors: bdfhjk [16:59] Logs for this session will be available at http://irclogs.ubuntu.com/2012/04/13/%23ubuntu-classroom.html following the conclusion of the session. [16:59] Welcome in the first Ubuntu Algorithm Classes! [17:00] Please don’t be afraid to ask questions. [17:00] if something isn't clear [17:00] I like active participants and questions are the important part of classes. [17:01] Classes will be splitted into 3 topics. [17:01] During first 20 minutes, I will talk about binary search, one of the basic algorithms. [17:01] Next 20 minutes we will spend on learning hashing methods, a more advanced topic. [17:02] Then, we are changing topic to introduction to artificial intelligence and usage of FANN library in practise [17:02] Let’s start with simple example. [17:03] Your friend has chosen randomly a number between 0 and 100. You have to guess it. If you tell him a number and this number isn’t same as his one, he tells you if the number you tried is greater or lower than his one [17:03] How many questions do You have to ask to guess your friend’s number in the worst case? [17:03] 50 [17:03] no [17:03] that isn't correct [17:03] log 100 [17:03] 100 ? [17:03] log100 [17:03] 100, if you're an idiot [17:04] The correct answer is 7. Each correct, smart question will reduce the set of possible answers to a half. [17:04] so, In general case, we need only log2K questions to guess [17:04] 100* log 100 [17:04] such an small number? how could this be [17:04] where k is the upperbound limit to number [17:04] This is an idea of binary search [17:05] We can use it to solve each problem, where it is easy to test, if correct answer is below tested value or above it. [17:05] Every time we ask about state of the element in the middle [17:05] Depending on in which half the correct value is, we call this algorithm recursively in suitable range of initial set [17:06] How it work? [17:06] Below I’m presenting algorithm [17:06] int p := 0 (number of first element) [17:06] int k := n-1 (number of last element) [17:06] while(p int sr = (p+k)/2; [17:06] if (try(sr) = success) p := sr; [17:07] else k := sr-1; [17:07] if (k-p = 1 && try(p+1) = success) p++; [17:07] else k--; [17:07] } [17:07] Please pay attention to line [17:07] if (k-p = 1 && try(p+1) = success) p++; else k--; [17:07] This fragment prevents the infinite loop [17:08] It is clear now, how it work? [17:08] On notes, we have also a another algorithm [17:09] Shorter, but may be more difficult fo understand [17:09] where we can forget about infinite loop [17:10] This algorithm work, when value is an integer [17:10] so what if this is a float number? [17:10] harder to find [17:10] If You plan to use binary search to find a float variable [17:11] choose an EPS value, a very small number (typical 0,000001) and break loop when k-p < EPS [17:11] so sometimes it is impossible? [17:11] it depend, if we can ask questions [17:12] "One the which side of number x is answer" [17:12] if we can ask this type of questions [17:12] then it is always possible [17:12] if answer is a float [17:12] then we can reduce set of possible answer [17:12] for example [17:13] 0-1 [17:13] 0 - 0.5 [17:13] 0 - 0.25 [17:13] .... [17:13] 0.1345 - 0.1346 [17:13] and then we know [17:13] that the correct answer is about 0.1345 [17:14] I hope that is clear enought [17:14] it is a simple algorithm [17:14] but need some time to understand how it work [17:14] In the notes, I wrote about some practical usages of binary search [17:14] Often on job interviews for programming, one of questions is to write binary search pseudocode [17:15] and one of the common errors is infinite loop [17:15] I saw, that we have questions on discussion room [17:15] I wrote code as pseudocode [17:15] it is used only in books or lecutres [17:16] to present idea independent of language [17:16] jsjgruber-l84-p asked: Where will we find the ntes? [17:16] https://docs.google.com/document/d/1CHU5TgXegp6IkRQhx7ok9HFNMPVONiWojbkIgfdFJrQ/edit [17:16] here [17:16] raju asked: where we can use this / area of usage ? I am pretty new to this [17:17] Please look at notes [17:17] We can use this in finding value in sorted set [17:18] but not only [17:18] for example I often use binary search method in debugging [17:18] trying to find where I have a bug [17:18] I put assertsa [17:18] 'asserts [17:19] always in the middle of code, where I may find my bug [17:19] khler_marcus asked: Works this with characters too? [17:20] Yes, it work for integers, floats, chars [17:20] and any datatype [17:21] with '>' operator - which we can compare, that the first is below/under the second [17:21] I think that is all about binary search [17:21] Do You have any questions? [17:22] I don't see [17:22] so [17:22] The second topic is hashing methods [17:22] In hashing, we are going to reduce amount of data (something like compressing it), in a determinist way [17:22] We assume, that each equal part of data always have equal hash [17:23] There is no return way. We can’t revert hashing. [17:23] We lose our initial data, but we gain something more useful in some cases. [17:23] A hash is usually a number like 1231432241 [17:23] The most important thing is, that we can compare hashes and if they are equal - we know, that corresponding data is equal. [17:24] There is a small issue, which I describe later [17:24] but in the most cases we can assume that [17:24] We may use hashing in many areas, but today we limit it to hashing words / sequence of chars . Also we may store different information in hashes. [17:25] I will present hashes, which store information about letters and their position in word - so we will be able to compare words [17:25] raju asked: if we got enough time please explain more about bug finding by using this . [17:25] I will answer this question after classes [17:25] so please stay online here [17:26] " QUESTION : " We lose our initial data, but we gain something more useful in some cases" If we are losting the data means its loss only na , please explain more ," [17:26] To make it more visual [17:27] think about beautiful landscape [17:27] You can make a photo [17:27] one or two [17:27] then look at both [17:27] and say - that is the same place [17:28] but in photo [17:28] you will lose the real beauty, that can not be captured [17:29] but your photo is smaller than landscape [17:29] and You are able to put it in the pocket [17:29] or share with friends [17:29] so You losed some 'data' [17:29] but gained some capabilities [17:30] We have a few possible ways to implement hashes. I will teach you about one of the most popular storing information about position of chars and their type. [17:30] Words are combinations of chars, each char is represented by a small number (ASCII code). [17:31] Our hash is defined as s[i] + s[i+1]*p^1 + s[i+2]*p^2 + ... + s[j]*p^(j-i) [17:31] where s is the array which contain word [17:31] and every element in it is a char [17:31] noted as number in ASCII code [17:31] p is a small prime number [17:31] q is a big prime number [17:32] It is quite complicated, why they should be a prime number [17:32] I encourage you to think about it, but now I’m only saying, it reduces the probability of collision. [17:32] What is the collision? [17:33] Sometimes, two different parts of data can be represented by the same hash, and we may think, that parts are the same [17:33] Is said, that we define hash as [17:33] s[i] + s[i+1]*p^1 + s[i+2]*p^2 + ... + s[j]*p^(j-i) [17:33] but this may be a very large number [17:34] so we will store and calculate it modulo q [17:34] ooooooooooooyyyyyyyyyyyyyyyyyy [17:34] q is a large prime number [17:35] bdfhjk: sorry for mis type [17:35] The greater q, the more potential hashes we can use [17:36] but also we need more memory to store it [17:36] I ussualy use q = 10^9+33 [17:36] we have only 1/q chance for that (if we choose q = 10^9+33, then we may have one collision per 10^9+33 comparation - it is small enough to forget about it in the most cases) [17:37] two hashes are identical and parts of data are different [17:37] Now, how to use it in practise [17:38] For example, we have 10^6 big words, and we want to be able to fast compare lexicographically two of it in logarithmic time of length. We want to figure, which is bigger. That is useful eg. in sorting. [17:38] The first step is preprocessing - calculate hashes for each suffix [17:39] when we know value of hash for each suffix, we can use the formula h(s[i..j]) = (h(i) - p^(j-i+1)*h(j+1)) mod q to calculate hash of fragment starting at char number i, and ending at char number j [17:39] It is clear, why this formula work? [17:40] not really [17:40] could you please explain this formula? [17:40] obounaim asked: what is q? [17:41] sorry not this question [17:41] jsjgruber-l84-p asked: what do you mean by suffix, here? [17:41] suffix is a last part of word, beginning at specified position [17:42] for example [17:42] word: algorithm [17:42] sufix [17:42] m [17:42] hm [17:42] thm [17:42] gorithm [17:42] and algorthm as the longest suffix [17:42] fasked asked: Why we calculate j+1 at finish, but not j [17:43] That isn't true [17:43] We calculate hash for part i..j inclusive [17:43] but we store in h[i] hash for suffix starting on ith position [17:44] so if we want suffix for i..j [17:44] then from suffix i [17:44] we must deduct suffix j+1 [17:44] and we have i..j [17:45] because suffix from i [17:45] we can denote as [17:45] i..j + (j+1)..end [17:46] We also multiply the j+1-th suffix [17:46] by p^(j-i+1) [17:47] because we removing chars at positions starting (j-i+1) after [17:47] i [17:47] Lilian asked: Can you explain if this is the only formula for hashes or we can create one ourselves(but the main idea it to calculate everywhere the hash the same way)? ;) [17:47] No [17:48] We can create our formula [17:48] It depend, what informations we want to store [17:48] for example [17:48] if we are interested only in number of chars [17:48] then we can simply add numbers of chars [17:49] then words abc and cba [17:49] will have the same hash [17:49] There are 10 minutes remaining in the current session. [17:49] or for example use prefix than suffix [17:49] jsjgruber-l84-p asked: Does the suffix hash arithmetic work while it is all modulo q? [17:50] Yes, we can multiply add and deduct in modulo arithmetic [17:50] we can also divide, but we must use fermat's lemat [17:50] I can say more after the classes [17:50] about dividing hashes [17:50] and for calculating p^x [17:51] we can use fast_binary_power algorithm [17:51] fasked asked: everything which allows to access data for O(1) is hash function, isn't it? [17:52] The definition of hash isn't strict [17:52] Always, when we store informations with data loss, we can say about hashing [17:53] when we only reduce size without data losss [17:53] 'loss [17:53] then we only making a compression [17:53] Ok [17:53] so to continue [17:53] to compare two words, we use binary search to find first char, where there is a difference between two words. [17:54] and to test, it two parts of word are the same [17:54] we test it's hash [17:54] which can be done in O(1) [17:55] so overall to compare lexicographically two words [17:55] we need O(log s) time [17:55] where s is a size of the smallest word [17:56] Do You have any questions about hashing methods? [17:56] gang65 asked: What will happen if we use too small p number (ex. 7), for formula: s[i] + s[i+1]*p^1 + s[i+2]*p^2 + ... + s[j]*p^(j-i) [17:57] if we store chars as numbers in ASCII code [17:57] is hashing different than comparing the ASCII value of the characters? [17:57] then they will may be between <65.97> [17:58] If I remember this numbers correctly [17:58] but at least > 50 [17:58] I often use p=13 [17:59] becasue if this number is prime [17:59] then our hashes always will be different [18:00] because if we split they into prime factors [18:00] always will be different number of p there [18:01] FeSludge : I don't understand your question, could You clarify? [18:01] hashing allow us to make it faster [18:02] Finally, the last but not the least topic : Introduction to the artificial intelligence [18:03] This is not such a difficult subject (in practise usage), as it might seem at first time [18:03] Artificial intelligence can be used in a simple way to identify the language of the text [18:03] We can also guess the complicated patterns with a small number of possible outcomes [18:04] I present, how to make the first mentioned task [18:04] Firstly we must define the input and output of artificial neural network (ANN) [18:05] The input can be for example the number of occurrences of a each character type in the text, so we have 26 input neurons [18:05] Output neurons can be languages, that we are going to detect [18:06] These data can be easily counted by a simple program and saved to a file [18:06] Next step is to create the ANN - artificial neural network [18:06] This can be done using FANN library, by calling [18:06] struct fann *ann = fann_create(1, 0.7, 3, 26, 13, 3); [18:06] where: [18:07] 1 - connection rate (we aren’t pay attention to this argument now, for our purposes it should be 1) [18:07] 0.7 - learning rate - defines how aggressive will be the learning algorithm. It depends on our learning data, for this example 0.7 is a good value for relative small amount of training data. [18:07] 26, 13, 3 - our layers. First layer is the input. The last one is the output. [18:08] What is a (lucky) number 13? [18:08] This is the number of neurons in the hidden, middle layer. [18:09] In every problem, we must guess, what will be the best amount of hidden neurons - there isn’t any strict rule. For some problems it is useful to make 2,3 or more hidden layers. [18:09] The next step is to train our ANN using pregenerated file [18:09] fann_train_on_file(ann, "frequencies.data", 200, 10, 0.0001); [18:09] First two arguments are easy to guess [18:10] Thrid represents the maximal number of epochs - how many times we ‘feed’ our ANN using data from file [18:10] Fourth is only for debugging, after how many epochs we want to see report at console. [18:10] The last one represent, what is maximal acceptable error on test data, so we can exit training after reaching it. [18:11] At this state we have operational ANN [18:11] ready for usage [18:11] If we want to detect language of text, we must only : [18:11] 1. Generate frequencies in this text [18:11] float frequencies[26]; [18:11] generate_frequencies(argv[1], frequencies); [18:12] this function read input text and count frequencies of each char [18:12] 2. Run our ANN on this data [18:12] float *output = fann_run(ann, frequencies); [18:12] 3. Print the best answer [18:12] printf max{output} [18:13] what represent the biggest number from output values in output layer [18:13] remind: each value represents one language [18:13] in output layer [18:14] And that is all [18:14] Do You have any questions about the last part of classes? [18:14] Lilian asked: printf max{output} - Python? [18:14] no, I wrote in pseudocode here [18:15] ow, thx [18:15] in c++ we must iterate all answers [18:15] and pick the bigger [18:15] language with the bigger probability [18:15] any other questions? [18:16] At the end, I want to say some words about Ubuntu Algorithm Team [18:16] Our main goal is to help programmers in problems related with algorithms [18:16] You can use our launchpad page https://launchpad.net/algorithms-headquarter to post your problems (as bugs) [18:17] Thanks everyone for participation! [18:17] We thank you. [18:17] jsjgruber-l84-p asked: So frequencies.data might contain something one line per language containing the frequecy of occurance of each letter? [18:18] an example frequencies.data: [18:18] bdfhjk: what do you mean by "problems"? Pseudo implementation of algorithms? [18:18] 12 26 3 [18:18] 0.103 0.016 0.054 0.060 0.113 0.010 0.010 0.048 0.056 [18:18] 0.003 0.010 0.035 0.014 0.065 0.075 0.013 0.000 0.051 [18:18] 0.083 0.111 0.030 0.008 0.019 0.000 0.016 0.000 [18:18] 1 0 0 [18:18] 0.076 0.010 0.022 0.039 0.151 0.013 0.009 0.009 0.081 [18:18] 0.001 0.000 0.058 0.024 0.074 0.061 0.030 0.011 0.069 [18:19] 0.100 0.074 0.059 0.015 0.000 0.009 0.003 0.003 [18:19] 0 1 0 [18:19] ... [18:19] You can refer on fann article [18:19] linked in notes [18:19] fasked asked: We can post any problems or only discussed here to launchpad? [18:20] Becasue we don't have many problems on launchpad, I allow to ask any question now [18:20] If in the future it change, then I inform about this [18:20] generally [18:20] answers is for all algorthim-related questions [18:21] bugs in for problems related with concrete program/application [18:21] by problems [18:21] I mean task [18:21] trouble [18:22] issue [18:22] the thing, which we want to do [18:22] Here I used pseudocode [18:22] to describe algorithms [18:23] sometime I mix it with c++ [18:23] but all should be affordable [18:23] if not, please ask [18:24] http://www.surveymonkey.com/s/MQK9SKP [18:24] here is a link to survey [18:24] please share your thoughts [18:24] about classes [18:24] There are 5 minutes remaining in the current session. [18:25] now we have 5 minutes for any questions [18:25] not necessary related with today's topics [18:25] I remember, that I had to answer some questions [18:26] please remind them [18:26] Which topics you want to discuss in the future? [18:26] I have some proposition, not decided yet [18:26] one of them are matrix [18:26] how to calc sum i=0 to n=10^18 [18:26] interval trees! [18:27] 3^i (i+1)(i+2) [18:27] Please send your ideas in survey [18:27] online judge [18:27] You can train coding today's algorithms [18:27] solving tasks in online judge [18:28] bdfhjk: thanks for class [18:28] for each topic we prepared tasks [18:28] so if You want to train [18:29] try solve they [18:29] some are easy, some are hard [18:29] binary search while debugging [18:29] suppose [18:29] You have a program [18:29] Logs for this session will be available at http://irclogs.ubuntu.com/2012/04/13/%23ubuntu-classroom.html [18:29] which crashes at some point === ChanServ changed the topic of #ubuntu-classroom to: Welcome to the Ubuntu Classroom - https://wiki.ubuntu.com/Classroom || Support in #ubuntu || Upcoming Schedule: http://is.gd/8rtIi || Questions in #ubuntu-classroom-chat || [18:30] then to figure where it crashed [18:30] You place a line [18:30] like printf("Here I'm still alive"); [18:30] and run your program [18:30] if You see this message [18:30] than you know - program isn't crashed before this line [18:30] in another case [18:31] You know, that bug is after this line [18:31] so you can ask questions like in binary search [18:31] bdfhjk: b-tree or fractral-tree ? :) [18:31] and every time place this line in the middle of unreliable code [18:31] Lilian: b-trees are interesting [18:31] but I think [18:32] fractal* [18:32] are not often used [18:32] are used primarily to read from disk [18:32] find an information in 3 or 4 asks [18:32] from a big database... [18:33] yes [18:33] If You are interested [18:33] please read about it in Cormen book [18:33] Introduction to Algorithms, Second Edition (9780262032933): Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, Clifford Stein: Books. [18:33] bdfhjk: nice, thx [18:34] there are very well described [18:35] bdfhjk: I went to a job interview and I was attacked with a bunch of algorithms to explain... [18:35] algorithms are very useful === yofel_ is now known as yofel === dduffey_afk is now known as dduffey === Resistance is now known as EvilResistance